{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mnist手写数字识别\n",
    "\n",
    "这个作业中，我们使用tensorflow来实现一个简单的手写数字识别的网络，并用这个网络来做个 简单的识别示例。\n",
    "本作业中，需要参与者应用视频中学到的知识：dropout，learingratedecay，初始化等等，将网络最终在validation数据上的得分尽可能的提高。\n",
    "\n",
    "## 基础作业\n",
    "\n",
    "### 问题描述\n",
    "运⾏课上给出的notebook的代码： \n",
    "https://www.tinymind.com/ai100/notebooks/74 \n",
    "给出代码的运⾏log截图并提供⼼得体会⽂档描述对整个模型构建及训练过程的理解。 \n",
    "\n",
    "### 批改标准\n",
    "- 代码不作为评判标准，如果运⾏正确，则认为代码没有错误。 \n",
    "- 没有明显报错的正常的log输出 60分。 \n",
    "- 对模型结构的理解10分。 \n",
    "- 对模型训练过程（梯度如何计算，参数如何更新）的理解10分。 \n",
    "- 对计算图的理解10分。 \n",
    "- 解释这⾥的模型为什么效果⽐较差10分。\n",
    "\n",
    "## 进阶作业\n",
    "\n",
    "### 问题描述\n",
    "以课上给出的代码为基础，通过适当的改造，修改初始化⽅式，增加正则化，调整神经元个数，增加隐层等，将这 \n",
    "个模型的验证集validation准确率提⾼到98%以上。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 导入用到的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# 设置日志打印级别为info\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "\n",
    "# 关闭ssl验证\n",
    "import ssl\n",
    "ssl._create_default_https_context = ssl._create_unverified_context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 导入数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./train-images-idx3-ubyte.gz\n",
      "Extracting ./train-labels-idx1-ubyte.gz\n",
      "Extracting ./t10k-images-idx3-ubyte.gz\n",
      "Extracting ./t10k-labels-idx1-ubyte.gz\n",
      "(55000, 784)\n",
      "(55000,)\n",
      "(5000, 784)\n",
      "(5000,)\n",
      "(10000, 784)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "# 导入数据集\n",
    "mnist = input_data.read_data_sets('./')\n",
    "\n",
    "# 查看每个数据集的shape\n",
    "# 训练集的images和lables的shape\n",
    "print(mnist.train.images.shape)\n",
    "print(mnist.train.labels.shape)\n",
    "# 验证集的images和lables的shape\n",
    "print(mnist.validation.images.shape)\n",
    "print(mnist.validation.labels.shape)\n",
    "# 测试集的images和lables的shape\n",
    "print(mnist.test.images.shape)\n",
    "print(mnist.test.labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面结果可以看出：\n",
    "- 训练集图片有55000个，每个数据是一个28*28=784的一维向量\n",
    "- 训练集的labels也有55000个数据，对应着上面每一个图片的值\n",
    "- 验证集的图片有5000个，每个数据也是一个784的一维向量\n",
    "- 验证集的labels也是5000个，对应着验证集每一个图片的值\n",
    "- 测试集图片有10000个，每个数据也是一个784的一维向量\n",
    "- 测试集的labels也是10000个，对应着测试集每一个图片的值\n",
    "由此看出，数据是完整的，没有缺失\n",
    "\n",
    "可以看到images里面有数量不等的图片，每张图片是28*28长度的一个一维向量， 所以用的时候需要先给它还原成28*28的二维图片。labels中则是图片对应的数字的值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.11  0.42  0.867 0.878 0.616 0.043 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.58  0.973 0.996 0.918 0.784 0.996 0.447 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.063 0.847 0.976 0.91  0.427 0.051 0.184 0.969 0.984 0.243 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.416 0.996 0.988 0.4   0.    0.    0.    0.584 0.996 0.514 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.624 0.996 0.663 0.    0.    0.    0.    0.549 0.996 0.514 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.506 0.996 0.616 0.    0.    0.    0.    0.549 0.996 0.514 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.404 0.89  0.553 0.    0.    0.    0.    0.824 0.996 0.243 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.184 0.973 0.984 0.161 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.494 0.996 0.824 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.098 0.725 0.996 0.702 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.502 0.996 1.    0.255 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.29  0.973 0.996 0.698 0.008 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.29  0.976 0.996 0.851 0.067 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.224 0.965 1.    0.843 0.22  0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.098 0.816 0.996 0.945 0.239 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.02  0.294 0.812 0.996 0.945 0.239 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.271 0.761 0.996 0.996 0.945 0.243 0.    0.    0.039 0.024 0.071 0.09  0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.424 0.961 0.996 0.996 0.996 0.773 0.651 0.761 0.757 0.839 0.808 0.914 0.961 0.42  0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.549 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.988 0.675 0.224 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.365 0.769 0.651 0.686 0.996 0.984 0.651 0.651 0.439 0.31  0.31  0.255 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]]\n"
     ]
    }
   ],
   "source": [
    "# 格式化打印\n",
    "np.set_printoptions(precision=3, linewidth=256)\n",
    "# 打印第一个图片的二维数组\n",
    "print(mnist.train.images[0].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上图可以看出mnist里的图片像素值都是0-1之间的连续值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHiCAYAAACtERYWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd81EX6wPFnUqgJXVF6DU0pKvaCAhbE3hvqoSjYOBX19Px5nGc5KyA2wIIV26Fgwwo2FESUIr1IB+k9JNn5/ZEw8501Gzab3dmUz/v18nXPZGZ35/hm8+x3ZmdGaa0FAAAkXkqyOwAAQEVB0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADwh6QIA4AlJN0AppZVSO5RSD0TZvq9SanvB41olun8oHq5n+RLD9Rxc0F4rpdIS3T8UX0V8jyo2x7CUUlpEWmutFxaUjxORT8KaVReR87XW70V6HEqHQq5nPRH5QETaikiqiMwRkdu11t8X9TiUDkVdF6VUHxEZLSLXaq1HBX7eTESWiEi61jrXU1cRpcKuqVIqVUQGi8jfRCRTRBaKyIla681FPa6s4NNfEbTW34pIxt6yUqqbiIwXkU+T1SeUyHbJfyMvEBEtImeJyHil1P78QS67lFK1ReRuEZmd7L4gLgaLyNEicpSILBORDiKyO6k9iiOGl4vnShF5V2u9I9kdQfFprXdrredprUMiokQkT0Rqi0id5PYMJfSQiAwTkfXJ7ghKpuAD1EDJH7H4Q+ebpbUm6VY0SqnqInK+5A9hoQxTSs2Q/E/O40RklNZ6XZK7hBgppQ4XkcNE5Llk9wVxcbCI5IrI+UqpNUqp+UqpG5LdqXhieDl650r+J+lJye4ISkZr3VEpVUVEzhGRSsnuD2JTMPf3jIjcqLUOKaWS3SWUXCMRqSkiWSLSXERai8iXSqn5WuvPk9qzOOFON3pXisgrmm+elQsFQ81vishdSqlOye4PYjJARGZorX9MdkcQN7sK/vffWutdWusZIjJGRHolsU9xRdKNglKqsYh0E5FXktwVxF+6iLRIdicQk+4ick7BMOQayf/yzeNKqeFJ7hdiN6Pgf4M3N+XqRofh5ehcISI/aK0XJbsjiJ1S6kjJ/52fIvlLhm4Wkfoi8lMy+4WYXSUiVQLl/4nIuyLyQlJ6gxLTWi9SSn0rIvcopW6W/A/EF4vIJcntWfyQdKPTR0QeTXYnUGKVJf9bri1EJEdEZorI6VrrVUntFWISXLcpIqKU2iMiW7XWW5LUJcTHJZL/wWmDiKwTkXu11l8mt0vxw/CyK1tEpiml7g/+UGvdVmv9l0/PSqmrlVKbCx4X8tRHRM+5nlrrSVrrTlrrTK11Ha31CVrrb/Y25nqWeoW+P/fSWncL2xjjPhH5reBx5WqIshz5yzXVWq/UWp+qtc7QWrfQWj+/t648vEfZkQoAAE+40wUAwBOSLgAAnpB0AQDwxOu3l3umXMAEcpJ8Hnon7tv1cD2TJxHXU4Rrmky8R8uXSNeTO10AADwh6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJxztBwAo/VJSTTh/ZBenavYpz5j4jCv7mzjty2mJ71cxcacLAIAnJF0AADwh6QIA4AlzugCAUietaWOnPP+huiZe0m1UWOtKJtrc0sb1vkxI10qEO10AADwh6QIA4AnDy6gwUttnmXhu/9pO3YJznzVxSNwjSFPEHov5zObmJh79RC+nXd0XJseln0BFldaimYl/v6eeU/fXIWXr2uXHmLj+t+tNnBe/rsUNd7oAAHhC0gUAwBOGl1GupDVu5JR/v+8AE7950vMm7lI55LQLBT5/hsStC3427VdroYkb3Pm60+rFCceZOHfFyug7jb9IqVLFxE2+UU7dMw2/N3Gqstdmzp6dTrvbTulj4rx5CwWlk0q33zae8686Jl7SI/Jwcosv/uaU2/T73cSh3Qvi2Lv4404XAABPSLoAAHhC0gUAwJMKP6e7+tajnbIKrBapssEWNrV1H3fgZPtl9CrjpySkb4jO4keOMvHcy5526oLLf4JLf0Jhnzc/2lnTxFO2t4j4WodWX2ri8zK2OnWrJswy8Ycd3CVJ2LfgPO7KMXZp1ocNXy+suYiIdJt1tonV4+4Sk8qLfi1xn9KaNTFx7tJlJX4+/NW84Z1MvKTHyIjtWk28ysSt+/zi1IV/C6M0404XAABPSLoAAHhSKoeX193gDvlu7phj4rEnD4/ra7WrNDVi3W6da+KaKVWdunVX7DDxqmHuP+MTa3qaeMOFNUycu3xFzP1EZBf0tEtIwneTcpf/2M+YT29u6bT7/JQOJi5quc/3Z1xs4jOfe9apCy4n+lC6Ft1p/MXCf9mDyed2fTpiu9ZfXmPiNv3nmTi0Y6nTzv1NiM78Ee51++Dkp0x80cu3OnVN/vVDDK+AhUOOdMu9nwmU7Hu0xefusqCsfrNNHMu1LS240wUAwBOSLgAAnpB0AQDwpNTM6c4faedS5vYa6tRVVunBkqcehb+ua//U6oHYrXul6TcmvvytbibedGkTpx1LEErg8INNeH1dO7f60c4DnGbB5T+ztjYwcfag/Zx2ix6xFzHr/mpOXd4cu61ccHlY+vPuhc8JTDStvNP9XkLD/zL/F04f1ckpf3Ppo4GSvQbLct3tHbP62qVZoZw9Je5HTo9DTTy2p/udkQ6BLQoRuz2n2r/vY88e4tSlKrtUzFkWdPVvTjsdKo1nBhUfd7oAAHhC0gUAwJNSM7z87ImvmDh8WPe/G1qbeN2ezJie/3/T7BBSk/GqiJbRWdHdfl55pNcbTl1wp6LXmk008eVvdHPabbrInojDcqJimjLThP3O62/i1NUbnWbu8p81Jlp5p7vr1JwT7NKQ00Ze69SlzrHxhr5296scPc1pF1ye1PT1P9x+hPcfsvZOd2h4/1Q7pLxL27o+A29z2lXL+Smu/dj+d/t+PbiS+7dnu842cfN3Njh15WOw04+69ywxccdKVZy6nnPOMHHWffZa5JWT4eRw3OkCAOAJSRcAAE9KzfDykIvON/E/O9dw6vZ/3+46k7fBHT6MVpZE3nkqFq3G23jUi72cujVj7A5JN9RabuLgULOISJt+dli02b0ML8dKT7VDzdEO41ZZ7+5pM2JLMxNXWrvdqVs82H4T+eUr7DB08AAFEZFp2fYzLIfY71u/rO8i1p0z7wITVxsbeThZpdk/Yapq1YjtwuUdbKcXnmz3UsR23aZdbeL9Z8+N+vnhurnhFxHrto6202y1Fkz20Z2k4k4XAABPSLoAAHhC0gUAwJNSM6erp9kTJOq6KzFK/VfzQzPcuZ6Xnuxt4hsGPxve3Hjjcrvz1t33Hh7/jlVAu85y/x03trW/4sF53Loz3XnbfjWXmrjzh+5yn8Mr28cFlwVNzXY/s/6zr11qlCruIdsonsz03SbeEVaXc/JhJq5z71ITv9Xis2K8wqRCf/p92DXd72F/O+CVN1sut6cJHV/lVxMfM+Ncp12tV3/01qfSgDtdAAA8IekCAOBJqRleBuJh1UXuLkdzTrDD+8ElPuGH3QfrgsPJ4XXBZUFXvHuj067F1+V/uUM8jRh5hlO+/nZ72MArLeyavOt/ONVp90JTe03TJOy0kRK6avz1Trn15Io19BlPm88MnxjIt3OceyhJhl6cuE6khP1+lIJdrrjTBQDAE5IuAACeMLwcByvuds9ODXXZFtXj6qfaodDckw516tK+mhbeHDEIfts4+BnT/XnRdf2Wn2Ti5f+wh28wnFwyOxqFXwOrqrLn2I5u+lVYrR0yvG2N/bb6xxO6Oq1yDrTvr4Unj4yqT/V+KflhKMh3YO2thf686obI1z1W2afZa7/+Wnv+8kH1Vzvttp1vf69yV6+RZOBOFwAAT0i6AAB4QtIFAMCTCj+nm9aimVNe2PdAEz9z8YionqNbFXf3oVQV3WeZRmkZJh7x0lCnbkDTY6N6DrgavFXJKV/Q0C5LOajGKhNfX/cHp13DwAHq4Z9FFz3UzsRVv54Sh15CRCTr+T+dcrucG6J6XKtX7UljoXmLTNw8151jX/zwUVE934CVx5i4zhvudyl0eGNElHZAfac8ss3rgVKGlFRqrZomPnvyAqfuosxhJq6ZEvm0qQ7DLzNxo/OY0wUAoFwj6QIA4EmFGV7efsERJv7zEPtZ49/njnHaXZy5KYZnL/lnlx5fDHTKWfJziZ+zIqr6gTv8m/2BjacFrlO/rv2ddtvut7vnfHXwW07dsf+yuxL9Nq2xiTmovmTy5i9yys3vWhShZdjjonz+tJ3RLf/5eVRnE9fLYRlYzNLTnWKTtJINKa8b4C7FPPu6iSbuV3NVWOvIQ8pB+2UWvkuWT9zpAgDgCUkXAABPSLoAAHhSruZ0VZcOJq413N3+6+Nm9mSSaJf0vL/DzknM2tUoYrsPH+nmlFOz7UKDK/9tT0v56zyEVWlNesS6iiKtsftvnLt8RcJeS0+d6ZQzAgfZXDDJPf1mbKuPTXzQNXYpV5N/MadbmqkiJn9zAzPDtedne+hN+ae3udvfjtjSwMRF/e1LrVfXxMv/1sbEMwc+E8fe5duyq4qJ94/7s0eHO10AADwh6QIA4EmZHl7+Y7D7lfJ7L7ZLPS7L3ODULcu1J0/M3VPbxDe9eY3Trtpqu8zgwInrTZz3+/yI/agpkQ+6XvCPwC4tYUMsS3K2m7jZB9ulItp1lj0lJrg0R0Tkwz/sdMGBZ8/x1qctjzVxyqHn7HRBTutd3vqBkrn6kgkR6y5YaKcQUif+ErEdope3eYtTfnOFPfmnX027du+YO39y2nW93x5if2HGl3Ht0+A/2zvlBjfbJUO5cX2l6HGnCwCAJyRdAAA8KdPDy7W6rnPKwSHl7r+f6dTlPHWAiYO7FjWTyDvQRLvzTbjQCV1MfHatFwI17mecjaHA5vxT3G/TlmfBbylf9NAnJv55azOnnc8h5eBm6uc/7A5LpggHm5cFqfvt55RbV14Yse36Z5uZOFOSs/F9ebf7JXt4TPajOSZ+9IDpcX+tHG3/Wref1NfEWf9wpxlz/1ge99cuLu50AQDwhKQLAIAnJF0AADwp03O6dfu6y2xa3WpPjmk5yJ2rTZNlXvokIrIpy+56ckyVyJ9r+s263MT1JPKSpPLmj0vtkpzgUoInp/dw2rWU+M/9GIcf7BRPe+kb26da7lxgKPDZNH1+dKeZwL8tJ7Z0ymdUs3Pz27W761SV9TmCxKrxhl0C+NN/7I57x1cprPW+5emQiQ/7+VKnrtK7dhloi1ft3/5kLQsqCne6AAB4QtIFAMCTMj28nLva/ap/y0Gl46v/G7oWPqgxZ89Op5z5TM1C25V3Db+2G6On35Jq4ls6f+W0e+Gm001cd7Y7PJj21bRCnzu1fZZTXtW9nokzTre/H18f/LLTLrgsKBT2WTTrk+tsPPiHQl8XyXfl4HER65bkuNc0/YvCf3/gX9vvrjCxmpVp4ubDZjvtdJ4dXt5/29zEdyxBuNMFAMATki4AAJ6QdAEA8KRMz+mWFqfM2uqUx9Z6OlCyWz1eOftKp13tT6YmslulV2DLy2NmnGvirw5+y2l2/V1PmTgkIadu8LpDC33qM2u+6ZS7VLaPSwl8xgx/vuDnzzbv3uDUtH/Ubh1XGpcgIF/d1MgndT22+pSwn2xObGcQUftnBzjlZg/ZbXl1rn2HxboNb2nHnS4AAJ6QdAEA8ITh5Tg4v8YMp1wtJcPE83PsocnVhtfy1qeyota1e0w8eJw7ZPxgffvvmqOdKrl//19NHBJbGX4iUHD5z9o8ewD9MxuOdtp9NvwYE7d+wd3NjCHlsm9PKHXfjZAwD7TobOLG4i670+GNyznudAEA8ISkCwCAJwwvx2jdADs8WT/V/Rbykhz7LcpLHhxk4nqfuMOWEMldvsLEv53R2Klr9d/Cv6EsIjKn2ygTHz/jQhP/ubFGxMe0GmIHivXUmU5dXeHalGcjm33olA99/O8mbnnbj+HNgYThThcAAE9IugAAeELSBQDAE+Z0o6QqV3bK511vT8TZFtrj1PWa0t/ETZ5nrjBauStWOuWWl62M0FKkt9j53hqyKBBHVtGWJlQ094y5zCm37fOEjdPd96+E3KVlgC/c6QIA4AlJFwAATxhejlbIHZx8dfyJJv7kt25OXZO3WYIA+Nb0/9ypnFv/76iIbVuyRAxJwp0uAACekHQBAPCEpAsAgCfM6UZJ57jLgprdw5wQAKB4uNMFAMATki4AAJ4ordmnBwAAH7jTBQDAE5IuAACekHQBAPCEpAsAgCck3QCllFZK7VBKPRBl+75Kqe0Fj2uV6P6heGK4nj0KrmdIKdUj0f1D8fD+LH9iuKaDC9prpVSZ3GeCpPtXnbTW9+wtKKVGKKXmFfwhvirYUGv9gtY6w3sPURzh1/MkpdQvSqmtSqnFSql+e+u01l8UXM9lSekposH7s/wJv6adlVLTlFI7C/638946rfV9ItIhKb2ME5Luvv0mIgNE5JdkdwQlo5RKF5GxIvK8iNQUkYtE5AmlVKekdgwlwfuzHFFKVRKRD0TkNRGpLSKjReSDgp+XCyTdfdBaP621/lJEdie7LyixOiJSQ0Re1fmmisgcEWmf3G4hVrw/y51ukr898RCtdbbWepiIKBE5Kam9iiOSLioMrfVaEXlTRK5WSqUqpY4SkaYi8l1yewagQAcRmaHdXZtmSBkfUg4qkxPRQAm8KSKjRGRoQbm/1np5EvsDwMoQkS1hP9siIplJ6EtCcKeLCkMp1VZExohIHxGpJPmfnu9QSp2e1I4B2Gu75E8BBdUQkW1J6EtCkHRRkRwkIvO11hO01iGt9TwR+UhETktyvwDkmy0iHZVSKvCzjgU/LxdIuvuglKqklKoi+ZP56UqpKkop/t3Kpuki0rpg2ZBSSrUUkd6SP2eEMoj3Z7kzUUTyRORmpVRlpdSNBT//Knldii9+OfftMxHZJSJHi8iIgvj4pPYIMdFaLxKRv4nIMBHZKiKTROQ9yZ/jRdnE+7Mc0VrvEZGzJX8KaLPkv1/PLvh5uUDSdWWLyDSl1P17f6C17qa1VmH/TRQRUUpdrZTaXPC4UHK6jCIUdj3f1lofpLXO1Fo30lrfqbUOiYgopboXXM/6kv9pG6UL78/yp7BrOl1rfajWuqrW+hCt9fS9dUqp+yR/bXa2iJTJc2k5TxcAAE+40wUAwBOSLgAAnnjdHKNnygWMZSfJ56F31L5bFQ/XM3kScT1FuKbJxHu0fIl0PbnTBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADzxespQWbZkTEen/N0xz5r40j43OXWpX//ipU8AIlv0+JEmvuXUT5y6jy85ysShGXO99Qn7cKT9O7vkFveQnvknjDZxq4lXOXUtL/01od2KJ+50AQDwhKQLAIAnDC9HSS+r7pTrHlfVxBvbVHbq9vvaS5cQR9mndzXxxmu3O3XTu74e1XNcv+I4E3/3SSenrsXzi02cu3pNLF3EPqQ1bOCUh5/1kol7Vt3l1I0+opeJ685IbL9QtDUDjzbxgze+aOKTq+5w2uVoGw89fIxTN0zaFvrca2862ik3eMNOJeRt2FjsvsYDd7oAAHhC0gUAwBOGl6NUfYWKWHfARX845bznEt0bxEKlVzLx/Ce6OHUfnfGkiVulu9MFoSif/7lG39rHXPuNU9f54D4mbnQew8uJsOi6pk45fEgZyaMq2/fUpgsPceq+uf1xE1dTlaSkVvzDDilPvWGIU/f2DY1MPGzIeU7dfs9NLvFrR4M7XQAAPCHpAgDgCUkXAABPmNONg1256U655LMSSIR5T3U28fwznnHqUqSKiUOiJRr9lndzyqMaT4rYdlhnu8Th8bonmDhZyxbKo8bHrEh2FxDB4n/ZedzZfYaH1Ub3F/O5zS1M/Pyrpzt1DeUHE2fXtd/CSFepTrvLMlebuOtdTzh1V8itJk7k/C53ugAAeELSBQDAE4aXo1Tj9NUR67a85+6Es5/8EaElEi24LEjEHVKe3Ts4rOUOO63O22ni48fe7tS1GLvHxJUX2OU+ees3OO26vHWZiad1fc2p+2VXMxPrPTkReo/i2t37cBMPbfFUWG26IHmCy4Sqt99U7Md/sjPTKb93x8kmbvjRD+HNiy0r7G/FmH88ZuJTugy07a6bWuLXCuJOFwAAT0i6AAB4QtIFAMAT5nSLkNfNfs19fIennbpf99g5wfqvz3Lqot02EPG3+obDnPL8M4LzfPaavbClidPuf9f2NHHr73+M+Py5Rbx2dnbkOcTxK+3h3FW3LSniWVAcu+raa3pwJeZwk0mluelk0b/t38/fDwtfJlS44DK8dee5c7qVV0Y3t9rsI/sdjI5Nr3Lqph31gonDlxM1T7PLBmvMTdzvEne6AAB4QtIFAMAThpeLkFfZfibJUO7JMzna7loU2rbNW59QtP79PnDKKWJPh3poQ3sTTz4zy2mnlv4a1fOn1qhh4hXXHOTU3dHxfyaevsedZKh6CkPKyfR9tnt/kbm8qIkCxCK7h3ty1++XRzekfMuqY0y89nQ7rJu3YVVM/Uj9+hcTN/narRs770ATX5ixLqbnLynudAEA8ISkCwCAJwwvF2HpOXwmKWvywj5HBg8v+PjBbibOXBr5G8qS4n6rMe+ETibuPfxLE19fyx27Cg5lnz7v7LAnXRn59RCzttfPjqrdkBU9nXKlT+O7y1BFtfZme2D8gP7vR/WY4HCyiMiSE+x7NrSz/B8AQlYBAMATki4AAJ6QdAEA8IQ53SJkHsBSoPKk2po9+24k7hyuiMgnr42M6nHnLOxl4pTzdjp1eVE9A4prQP3gvLqK2G7eJ62dciP5M0E9Kt9SOrVzyg/fbHd46l51Z3hzI7jTVHBZkEhi53FVlw5OuVn6LxFaiizMyTZxzcWJW1LGnS4AAJ6QdAEA8IThZZQrC3bVd39Qc6kJX3xlmIkfXtvDaTbxj1Ym/vTwYeKqaqItod0m7vrR351WbW+zy1dCO3ZE22V40PR9dziZ4f7YHPeqOzxb1JBy0NT3DzZxww0lP4A+WvP6V3PKh1fWEVqKTNhhd6yr+sGUhPWJO10AADwh6QIA4AnDy2FSqtgzFY9tGHmT+pHrTgiUtiewRyiOOTe0d3/w3k8mPDDVDhMPbfC90yylgR3yCgWGk8Od+NQgE2c94g6TcY6yH8FdkNqkB69BFafdyrzA0GcuA8qxWn/dUSbuX/vxsFp7EMzqvF1Oza1/2F3ZmvxvrYkTfSXSmjc18aRTnwyrjfze/m5jq0BpfXw7FcCdLgAAnpB0AQDwhKQLAIAnzOmGSalV08RPNfgkYrtJ39kDzFtKESfWIOGyT+9q4uUXuzvJpBSxS1FQqgp8/tTu7Gz32eeauMEj/pY7IF9q/f2dcpdLZ5q4RkqV8OZGt7G3m7j1At6jsdpmp0glI6VyxHaPrTvRfdxxwXnRxM2Rhpt3gz2oPvg9jnCbAsv/RETWDG1p4urM6QIAUPaRdAEA8ITh5TC5zervu5GINPk0J8E9QVBKx7ZO+YAR9lD4UY2fN3Hw0Pr8cuHuWtPVKf9vymEmfrbnaKfuhTavmbjPhXbIMuNthiy9qFfbKY5q/GmhzbaGDRdmLuGewqdPvzjMKTeXyf5eXNlpJJ0a3UNuX3GaU67+7k8RWsYXv5UAAHhC0gUAwBOSLgAAnjCnG2b9PbsL/XmvuWc65UoTfzNx5HMrUBLr+9nt5ybc+5hTV9NZKhJ5WdBtq4808Sdf2TmnrCfdLT6zVttTRR478TKnLniI/cX32WVkH77tzjUiMfKqV4qq3cwc90SZA4awvMunA79P3labWy47wsRzL3w6qsf88L27ZayvpZ/c6QIA4AlJFwAATxheDvPsQa8HSva756u21nDaNchd4alHFce2i490ysEh5ZphOw/NybFLtp5c09PE84Z0cNrVfP9XE7fYbZcwuPtWuVIn/eaU2759g4l/u2CIiceefKPTLv2zn4t4VsQq8/HVUbXrP92dFmgksxPRHUTQ9O65Tnnt+Pg+f1qjhiZecEMTp+6ny4OnH0XeNevNbXZJaNZLm5w6X4Pj3OkCAOAJSRcAAE8q/PByWjN3mCJT2W88pqp0392p0NZ3dL+FHBxSHrujjlP30oWnmzj06+8mzgz7BmIsB8unVHWHsjscstTElQO/E6G06A5TQPGlNW5k4qyMZRHbXba0h4mbXrPKqePYer+OrbXQKb/f2k4X5S1YHNVzpLZrbeIFV9Zz6oac/5KJT666I+yRkYeUg0bfcJaJ02ZPi+ox8cadLgAAnpB0AQDwhKQLAIAnFX5Od/cot5yVbufz8gKHmWe87S4ZQuIFD6C/8+sLnbqsX6fG9bVS69U1cbWx7lztWy0+DpSYx/VhTa/GJh63/zinLlXZe4VNu+0uVCl73CUgKt3uZKVz9sS7ixVG61F2ydbgXp2duvv2s0vyrq6x3KlLHWf/fs7c2Uii0bn6JBNflhndUrFw43bYneJu/+Jip67tj3YZWSzf94gH7nQBAPCEpAsAgCcVcng5NauliW9rNi5iu0uW2J2Oaozxc8BxRVZvhnt0xKbQLhNP7TXEqev6/EATt/u/P0yct3ZdxOdPa9jAxDs6NXTqBg5908SnV9vi1AWHoZ7ebH93qn47N2I7JE5w2ufjtoH373y3Xet3B9j4Fj+b2ZdHuYuXmnjCsGOduoGD7b9r+K5xfWqstIVgHAc7tTtd8PRGO+z9zd+6mjjr5ylOu9LwHuVOFwAAT0i6AAB4QtIFAMCTCjmnu6dhTRN3r5odsd38t9qYuL7mQOxEyxzjzrsd32qQiX/r/5RTN7/3cyaefbI9M2jggosiPv/r7ewJUuHzT8HlSeHzPretttvZzb3JHnyttv0mSIwqG+1VWJS7y6lrmVa10MfsCpvnq7aae4p4q/PiZKf8f/27m/j6/SY6de3S47uNbvD7FK8OPc2pqzci2K9ZcX3deOO3EgAAT0i6AAB4UiGHl4ty/YrjTNzgzXkm5sQS/+q7Ywk4AAAgAElEQVTMtf/qz21u4dS1r7LCxN2q2KHhzzu8V8QzVolY89yWpiZ+8qPeTl3re6ebWO1mSNmHjHfsEr0LDxjk1P36j2dM/J/1bU383oiTnHYNhzMllGiLuu428V2tLnHrrjrAxKec+rOJHz/QnUbq8MqNJlZF/KFt+cYGE9f7fXLkhqUcd7oAAHhC0gUAwBOltd53qzjpmXKBvxeD4/PQO3HfqT+Z1zOtWRMTL3i4VsR2Dx3yvol/2NbKxOMnHOG0a3532RquSsT1FOE9mkzl7T1a0UW6ntzpAgDgCUkXAABPSLoAAHjCkiGUSblLl5m4+cXLIrYbIcGlRnaXo+ZStuZwAZQP3OkCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgiddThgAAqMi40wUAwBOSLgAAnpB0AQDwhKQboJTSSqkdSqkHomzfVym1veBxrRLdPxRPDNezR8H1DCmleiS6fyieGK7n4IL2WinFiWqlUEX8m0vS/atOWut79haUUmcopWYVXOgflFLt99ZprV/QWmckp5uIUvj1PEkp9YtSaqtSarFSqt/eOq31FwXXM/JZgUi28OvZWSk1TSm1s+B/O++t01rfJyIdktJLFIe5pkqpekqp75VSG5RSm5VSk5VSx+xtWB7+5pJ0i6CUai0ir4vI9SJSS0TGi8g4PjWXTUqpdBEZKyLPi0hNEblIRJ5QSnVKascQE6VUJRH5QEReE5HaIjJaRD4o+DnKpu0i8jcR2U/yr+l/RWR8efqbS9It2iki8q3W+jutda7k/wI0FJETktstxKiOiNQQkVd1vqkiMkdE2hf9MJRS3UQkTUSGaK2ztdbDRESJyElJ7RViprXerbWep7UOSf61zJP85FsnuT2LH5LuvqmwWInIQUnqC0pAa71WRN4UkauVUqlKqaNEpKmIfJfcniFGHURkhnY3G5ghDCmXeUqpGSKyW0TGicgorfW6JHcpbki6RftCRE5QSnUrGLK6W0QqiUi15HYLJfCmiPyfiGSLyLcico/Wenlyu4QYZYjIlrCfbRGRzCT0BXGkte4o+aNSl0o5+1BM0i2C1nquiFwpIsNFZLWI1BOR30VkRTL7hdgopdqKyBgR6SP5H546iMgdSqnTk9oxxGq75P9hDqohItuS0BfEWcFQ85sicld5+t4FSXcftNbvaq0P0lrXFZH7RKSZiExNbq8Qo4NEZL7WeoLWOqS1niciH4nIaUnuF2IzW0Q6KqWCU0AdC36O8iNdRFokuxPxQtLdB6XUoQXzf/uJyAgRGVdwB4yyZ7qItC5YNqSUUi1FpLfkzwOi7Jko+V+0uVkpVVkpdWPBz79KXpdQEkqpI5VSxyqlKimlqiql7hSR+iLyU7L7Fi8k3X0bKiKbRWSeiGwSkWuT2x3ESmu9SPKXIwwTka0iMklE3hORUcnsF2Kjtd4jImdL/nTBZsm/tmcX/BxlU2UReVpENojIShHpJSKna61XJbVXccQpQwFKqd2S/wWbYVrre6Nof7WIPCkiVUSkvdZ6cYK7iGKI4Xp2l/wkXFlEemmtv05wF1EMMVzP+0TkVsm/ntW11nkJ7iKKqSL+zSXpAgDgCcPLAAB4QtIFAMATki4AAJ543US6Z8oFTCAnyeehd9S+WxUP1zN5EnE9RbimycR7tHyJdD250wUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADwh6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJ14PPAB8S2va2MSbj2ho4tW99zjt+h8yycQDa8936g767moTh5ZWN3Grwb857UI7d0bux4EHmDh39Zp9dRsoV3K7H2riDR0qO3W79rdnMuhWO0x8Z6fPnHZ9a9r3zac73ecYNKKviRs88kPJOptg3OkCAOAJSRcAAE8YXka5smrQ0U75nmveNPE5GesiPi4l8PkzJCGnbsaxL9jCsTbstPsWp13T+yIPa1V+K8/EucdHbIa9lD2KdF3/o5yq/je9b+J+NVfF9PQjtjQw8ftnHmni0NIVTjud405DIHpbLrf/rl89PMzElZWbdkJS+JG/KeIeR5ujbbvuVd2pnO9uftzER6feZuJGD5W+oWbudAEA8ISkCwCAJwwvh0np1M7E826tauIrOv/ktLupzhQTd398kFN3wJDSN6RRnqW2zzJxcDhZJPKQ8p952U75j9xqJs6TdKfusEp2iDE1MOz52zVDnXZdt9rh5gMfd38Hjq2zyMQTpEahfarwUlJNuPyeI0w88/rhER+Sre2w/apc95pWCYxO7p9azanrW8MOI/ed+K6Jh25q5bT7svdBJs5duixiP/BXW8/ebuJ0Za9t+HDystxdJr5nxZkRn++nuS3s81V3h/2/O+ZZEx99tl1VsPwJ91vOOtv9HUkG7nQBAPCEpAsAgCckXQAAPKmQc7qqsh3nX9PvUKfup7vsPN22kJ03OHLM7U67bzrbuZ8TLp/q1M0bEpduIkpz78owcfgcbvAanvjztSauP7SK0y514i8Rn3/9dXbJSu8B35j47nq/Ou3y3Okjx3cbWwZKf0ZuWIGtHBTtPG6uiTu9YefRW9wx2WmX2q61ief+I9Opm3XScyYOLmG5pfZC98U+tOEX3Zo7VXnrN0TsI0SaXbvSxAM+tevkZm08wGlXO7DyLm/+IokkSzZGrDviub+beP4Zdn638203Oe0aPZj879twpwsAgCckXQAAPKkww8spVexw4twhHU288Ax3GOupzXZI6p3Bp5q45dthQ1dZdrhwRsvOTp0+w65VSNtplzSkfTmtuN1GFP533LOBkvs5csAfdglCg3N+j+n56z1vr/1X6+yWVHcP/7Ww5oWa96n9vWrE8LKIiKg0989PpWOiG6496H92yLB12JByUN6cBbZdH7fuuH52TPORO0eYuFuVHKddcLj5y8yD3SdheLlIeZs2mXj6SDtFU2uRu2wnb37kqZ1ope4o/P6xQ695TnnLgyV+qRLjThcAAE9IugAAeELSBQDAk3I7p5tSzd32beUbTU28sKtdLvDEptZOuwk3nWDijK9/jPj8wa+2V9u01akbOHmiiUetsV+V3/LlPjqNmBxcyW7bGL7F3NT5dplHlpR8Di5zlp2P/W63u+yo7uzc8OaGVhGrKqzUJo2c8tRD3yy03VObWzjlts/ZucK88MZRqjfCzgWPvfYwE3drEHmOGLGrOyo5/6696/3mlF+XRhFa+sOdLgAAnpB0AQDwpFwNLweHlOc+fpBTFxxSfmxjGxN/c2Z7p13qkuJ/fX35Ve4QdfeqE0y8cT/7fK/U6ui0y9u8pdivhb86cdZ5Jv78oLedutHdRpn4AXGXdkUrt7vdtWy/++20Qos09/rVu22JiXd84D6HKvyc7gpt6UUNItZt13ZZyZgHT3Xqav4eedonFouvambi78e7p4kdUzlk4gX93P62uNfuuKRzI08tIP6yT+vqlK/qObHQdu+v6xL2k+Qv1+NOFwAAT0i6AAB4Uq6Gl/+8rJOJF575tFP30U67Kf43Z3Uwce6SpSV+3T01I48dztlth6QYTk6MjIH21/jZd92h/n4155t4/jOHm7j9f1c77daebL/VeMaNk5y6PrXsIRgN0oKnGrgnHLzSYryJe/dyN1rPrcr4sohIat06Jr7zyrcjtnt3m/3Wec3X4zucHC5vtt216MoJ/Zy6hWfaaak5fdy/Kae/F9jm6udZielcBZZao4ZTXnuJ/bt93UB3/qZvjRUmXpq7y8QbHnUPqajC8DIAABUHSRcAAE9IugAAeFKm53TTGrpf4b9j0BsmXpm306l76L4BJq6xuORzRGktmpm492k/RW6IhAueJvPq0NOcuv732bq5ZwXm5M5ynyMl8PkzJCG3Ugo/nf7ONUc55fHf2J2N2s5c4dRd94g94WjCve5cVUWiAqd9XZa5Lok9KVyNuWF/Es8svJ2IyLzr7f+XrGsS1KFyKKWzu0xzVbdaJt7axi69uvYY97sVg+p+XcSz2i3fenx8q4mzxk+JsZeJw50uAACekHQBAPCkTA8vh+q6w3TnVbcbof97/RFOXY03ij+kHDxke+XAw526u659y8QXZyT/a+gV2a6z7LU57rqpcX/+vn/0NPGftzYxccqMhU67Vjvt7xj7E5XM15vaBkqbk9YPRC/twAOc8pWT7CEHp1RbY+J0cYd801VqiV/72Nvt9GHWW/H/GxBP3OkCAOAJSRcAAE/K9PByUc6sMd0pf9jvFhOn74y8O9DG0+1uJh8e/YyJW6a5QyLv77DfuGs17nqnLriLzdSNTQM1q4ruNKK28Wr7zeELb/vMxANrzw9rGd3nyuAQV/un3d2kGj/wQ6BkhzrDv+NclBRVnNbl1+JrmkXVbtYY+w3X+vJDES1RWuja7nTfOdU3BkqVEvrazoEioVhPWfaDO10AADwh6QIA4AlJFwAAT8r0nG5o5jynnPW2/dr4/Aufceqm3OeeEBKNT3fVNfHZo/7m1DV5ZJqJ27bZ6j4wsIvNgql2TrcFc7oxS2va2Cnfe/doE59WbZuJw3eT2phnD0M/c4a9hq8c9LLTrlW63XUqbXeJulqokObzrYjI7qZ7kt0FJMpqd+nkEdMuNXGX/Vea+NuvDnbaVV2rpDC76rvfvfn3eWNMfF7Geqeu190TTfyxdDNx5pjEnlAVC/4SAADgCUkXAABPyvTwsmh3+KHV3+1QwuFzb3DqQr02SWE2r8t0ys3es3GlT+3OJo3Dli0EX1nPmOvU/Wf9QSa+/BS7afcPdyT2a/PlTWqbViZ+aMJrTl2bdLvEZ1muHULu9dogp12rZ/4wcZ2VdjlR71fd34+5J42y7U4JmwZ4MrBjTozLEV5441QTN2IJDMqhvE3u39j9zrTl4PEfzWWyxOLVp+wug0+9VM2p++pgu0PgpGtb24q3w3a7KgXLibjTBQDAE5IuAACekHQBAPCkbM/pFqHe82HzBs8X3m7/OLxWat06TrlLNTu3PG1n8zi8QsW04L4MEwfncEVEvthl5+L/9cDNJm72knvdI5320+oKd5vQ8yadbuIJHd5x6o4cYLcQ3X94bPOxjR5kHndfVuftNHGNZaX/nKbqC/mOhk+5q+1JRRmnunW3TT3WxB+3fd/ER157o9PuL3khCbjTBQDAE5IuAACelNvhZZ90Q3eQ+vRq2018y7f2NJws+dlbn8qDl498MWLdo7dcYeI6H5V8yGjRpy1swR2RkmsGjDfxuOF1BYmRmWKnELJr2Lhqgl83tZ1dYnL5tROiflzT0YtNXPoHw+MjtXZtp6z32B3GQjt2+O6O8ek3XUz85MV2KuecG7522n37fBVvfYqEO10AADwh6QIA4AnDy3GwsmediHVp69M99qR8SQ3s+5US9vmw8obs8OYl0uxlO1T4Wh/3cIVjqi408Uf1skyct35DXPtQEWTODnzj9xS3LkPZQyeOusXuBjfnlcT2qeHLdgeyW2sviNiu3Wh3F7MWf06N0LJ8SWvcyMTtP1jp1H34gZ0+azI4sd/QV5Xt78eyQYc6dXf0ej+8eX6fKq0P+0mjQtv5xJ0uAACekHQBAPCEpAsAgCfM6cZBdm2970Yottc2HG3iLg2+c+qW/t3GLR5qb+LQr7/H9Fo6154+siXPPcGkXSX72XTdOXZOt+7I6Jcqbbv4SBOXxoO1fWk8Zqkt3Bq53cHV7Lk0c+SAuPdj8cN2LvLthk8Eaio77UZusfP7rZ5c6NTl5VaMhUJbDm9o4ofrj3Pq7r7mexMfWu/vTl2bUVuL/VqLL6hl4pzaIafu/h7vmvjCDHf+OEWUiYOPeub+8512NSX57z3udAEA8ISkCwCAJwwvo9T67ItDbKGPO7w849gXTLzqA7t86PF13Z12n3zbRaIx9twhJg4/XGF6tv1sut/rv5nYHfwq2vn//MzEE8bUKMYjyxcd2LVo6KZWTt0tte3w7SWZy0z8wCu9nHZtHrMHI4RmzI3qdbdfcIRTnn75kyauGliqFBxOFhEZd56d4sj7M/JyovKs+spdJv7P+oOcun/Wm2Xieec+49SlnBsc8g0u/1NOu2Cd8/go24mIrAsclnHMB7eZOOtd92CT0jARyJ0uAACekHQBAPCEpAsAgCfM6SZAqrKfZWrPTmJHyrhWQxaZ+KeL3O00j6icY+JGafYcmsfDlhY9fpFbjiRF7POHwmZrP9nW0dbt3CmxGDnnGBM3kZkxPUd5kLd5i4m/7O3OD8qHNgzO7y7oPspp9urhdgnRf8e4S0KCLjv3KxvXfNypq6qqhTcXEZGnXjvLKTeak9itDcuEH2eY8Jtbj3KqTv6HnZf/X9u3nLrgtp7h87NB7nIfO+v6+jb39LbzM+x2nR0+HeDUNR1rn6P1Rz+ZuDTM4YbjThcAAE9IugAAeMLwcgLkaTs8WXvO9iJaoih5a9eZ+OFTz3Pq5g3Yz8T9un9p4oF1YtuRqu+yE008dYI77NnihWWB0gqJRZMLKu6QciS5S5c55TeGBo4duiUQ1nZ3groic42Nrx0e5au5w8kvb21g4vfOP8HEjeb8JIgs7ctp7g/sW0/OPOMWp2rVJfaA+ynH2eVE58+72Gm3/kN78o8KzOw0eN1dDja6kx36z/rq56j7XNpwpwsAgCckXQAAPGF4OQGC315GfOTNX+SUWw205a+keiDuGuMr2M3Zm4j7jdWKsa198gUPkPjs5Xom/qJZZ6fd3Bvtt1qPPdxOJ3w3pb1E0nbEJqccmr/ExDpnXvE7i7+oMn6KU24x3sYXi93ZK03caYUDwsp75YWV077aWKL+lRZkBwAAPCHpAgDgCUkXAABPmNNNgEU5dplQ6ma7g1H4HAWAwukcu9wkb8Fip671Lba8NvjzIg4o572H0oI7XQAAPCHpAgDgCcPLcdDsn5Od8oB/HhsouUtdAAAVF3e6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8UVrrZPcBAIAKgTtdAAA8IekCAOAJSRcAAE9IugAAeELSDVBKaaXUDqXUA1G276uU2l7wuFaJ7h+Kh+tZvnA9y58YrunggvZaKVUmzw7g28sBSiktIq211gsDPxshIieISGsR+ZvW+uVoHofkC78uSqksEXlURI4WkVQRmSoiN2ut5xX1OJQOhVzP40Tkk7Bm1UXkfK31e5Eeh9Ijwt/cziLygoi0E5E5ItJXa/1roL6ZiCwRkXStda7XDscBd7r79puIDBCRX5LdEZRYLREZJyJtRKS+iEwRkQ+S2iPETGv9rdY6Y+9/ItJbRLaLyKdJ7hpipJSqJPnvyddEpLaIjBaRDwp+Xi6QdPdBa/201vpLEdmd7L6gZLTWU7TWL2itN2qtc0TkSRFpo5Sqm+y+IS6uFJF3tdY7kt0RxKyb5B85O0Rrna21HiYiSkROSmqv4oiki4rseBFZo7XekOyOoGSUUtVF5HzJvzNC2dVBRGZod95zRsHPywWSLiokpVQjEXlaRG5Ndl8QF+eKyHoRmZTsjqBEMkRkS9jPtohIZhL6khAkXVQ4Sqn9ROQzEXlGa/1msvuDuLhSRF7RfDO0rNsuIjXCflZDRLYloS8JQdJFhaKUqi35CXec1jqqZQoo3ZRSjSV/LvCVJHcFJTdbRDoqpVTgZx0Lfl4ukHT3QSlVSSlVRfIn89OVUlWUUvy7lUFKqRoiMkFEvtda35Xs/iBurhCRH7TWi5LdEZTYRBHJE5GblVKVlVI3Fvz8q+R1Kb5IHvv2mYjskvy1nSMK4uOT2iPE6hwR6SoiVxdsmrD3vybJ7hhKpI/wBapyQWu9R0TOlvxrullE/iYiZxf8vFwg6bqyRWSaUur+vT/QWnfTWquw/yaKiCilrlZKbS54XCg5XUYRnOuptR5dcP2qB9d3aq2XiXA9y4C/vD9FRLTWbbXWL4Q35nqWCYX9zZ2utT5Ua11Va32I1nr63jql1H2Sv3dCtoiUyfl7dqQCAMAT7nQBAPCEpAsAgCdeT2nomXIBY9lJ8nnoHbXvVsXD9UyeRFxPEa5pMvEeLV8iXU/udAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPvJ4yVNYsGH2Iief1GOnUnXTjABNXG/uTtz4BQEWQ2qGNU156bl0TH9ZrllP3StNvTJyj86J6/u439HfKVd+fUtwuxoQ7XQAAPCHpAgDgCcPLRdH2DOKQhJyqld1t3Hqsrw5hr7TmTU28/JyGJt6Wleu0a5O10sTj24wzcdaH1zvtGk2wnz9rTF/j1OntO02c9+efJlZp7ttn1c2Hmzi3qtvfJo9Ns8+XnS0A/mrrpUea+PS7Jjp1Y+vOjPi4HG3fv+F/qyN5dshQpzxoXh8T581ZENVzxII7XQAAPCHpAgDgCcPLMWrZbpWJVeXKTh3Dh/G3ZuDRTvnnQU+ZONrhpGCr+b2fc+t6R36Ot7YdaOIX/36OiVcd5759Zl7pDlcFnTHxWhOr73/dV1eBciulShWnvOhfXUw8+4rhJo72fR2rrPRKTnnOLbVt3fXhreOHO10AADwh6QIA4AlJFwAAT5jTjdHHbd838VkZPZ26POZ04yK1VXMTj77lybDa4v/qjt2+v4nPy1gf9eMuylxt41HPmDgl7DNrcAZqerZbl7pld6HtKpq1N9u5+a2H7S6iZWKlV7ZLy2Yd+1LEdr0bHuqjO+Wfsssvg3O4IiIzrxgWKJX8PrD92zdFrPv9wqci1j104jsmfunw3rZiSuSlSrHgThcAAE9IugAAeMLwMkqtVb3sUp12lSJ/Pjxp5kUmrn5/jYjt0ldvNvELDWo5ddl17fKBAY+849Sdk7Fu350VkVl7tIkH3TbAqas2i0MxRER2HGl395pzwsiI7YJD97EuHYn2OYI1r21tHNNr4a9Cx9lh5MX97M9/P2lYIa3/6t3tBzjlf35nl+s1Huf+Paj6gT2soJX8aGLVpYP7pBdGfr3g+3xYi+omzozzOQjc6QIA4AlJFwAAT0i6AAB4wpwuSq1jr5gWsW513i4Tr51Z38Spp0V+vvo/23nbtYeluq/Vwy4LiHYON9yHWzubuNpY5nAL03rAEhOfm3mOU7fkqiYmzq5tZ1qVlpiE6u0x8Zwez0ds1/ZjO//e7o6FYbWbYnvxiiiwLEgkfB53RFRPcca8M00cunc/py7r+59j71spwp0uAACekHQBAPCE4WWUWh/93MnEj5zxrVPXJC3DxHMuHS5RudqG6codXs7ReYGS+1l0fWAo+7h3bjfxxAsec9rdXc8OUXe78AanLuPtHwUieZu32EIwFpHG96+I62ttv9AeiC493LqFOXZHqnaPbrT928RwcnEETwwK32kq2qVBP2Wnm1iftNLESlYW1rzM404XAABPSLoAAHjC8DJKraz+diuYQ+r2depmHvOyiWPZsSgn7Bux43bYA6yHLunu1KUMrWfilh/bYeLjqt/qtJt7xtMmXtUzz6nLervYXUQJre69J2Ld4BV2Q/u8+Yt8dKdc0u1amtg9uCCydl9e55RbjrDv3xT5NT4dK8W40wUAwBOSLgAAnpB0AQDwhDldlAktBrvzc9063BChZWxq/bzGxFUXLwmrDS/v28FZy51ydiydQoks6D7KxKGw+4tpU1qbuJVs8Nan8mZl95omTiniHm7sjjombj08x62M8yHxRQn28a/LBm2s3c214twHAADgBUkXAABPGF5GmZA3e55Tzpgd3+fP3XeTv2iTFXnHnJnz3cPQs2RNhJZIlJDoQOwuK4v1EIWKLq1xI6d86qWTTVzU0r07v7rIxFlT4nwqfBFW3OuWg30MXzZ45VK7bVntj343sbv4r+S40wUAwBOSLgAAnjC8XJTAGFT4N/PCv/mGiiGnx6EmntDGPSN0cmDj9jbP7HTqGM1MvF1nHR72k8jnMefVsd+gXfyGPQf50KbLnHYDD/zcPkbcr7Re++KNJm78nx+K09Uya+Nx7vDyf+qPjdi256wLTdzujrkmjvdwbbilb3U08YudX476cYuea2viWlsnF9GyZLjTBQDAE5IuAACekHQBAPCEOd2iBLYlCf86fPDr5nMebOnUZV23UVB+pGRmmvjBEXYeN3xe/5vtdk5IT4/zmqZyKLX+/k5529HNTbyrjr0fSDl3fVTPN7rDkLCfVI7Ydu7Jz0X1nH3/6GniaZ+2d+qaPWFPxCn+OVdl04Yzd+67UYHlK+qaOGtr8Xd1i9UdHT8z8WGVI88g9112olOu++lCEydy3pk7XQAAPCHpAgDgCcPL8VCpogwuVQypdes45e1v2E3du1SOvKPNi5NOMHFr+SkxnSvjck4+zMSZ9y516sa2GG7i4BK9onY6cqXvu0mB4LDxn7c2idzwxxkmbCLusqCK+K6/u/OnTrmoQw6y+v6c6O4YWz+xU3x9agSXikXu3+8vdnDKdf9M3DKhIO50AQDwhKQLAIAnJF0AADxhThcIs7xvW6f880FDC233n/UdnXK7J9eaOJZTiyqCP06zf3ImtJjg1L2+raGJN+dVM/EHqzo57dZ93VAKM6zv8065e1W78KPrL5c4dXV6zw+UNhfdaRh52r1Pi36+veRSa9nvVix8rqlTN7vjS1H1qf3bN5m41Ug/c7jhuNMFAMATki4AAJ4wvIwKKbjLlIjI2tcbmPi9To+Gta5komGb7NDzhEeOc1rVXPxj/DpYTtWaY3d5y/r4eqeu3SA75Ju3eYuJK8kfTrtGYeW9frvUHXI8vsqCmPuJ5Aue6CUiUv9+ez3HNnkhrHXh949f7HLf521G2t0CE33aUSTc6QIA4AlJFwAATxheRqm1ZuDRJq7Uw930/vH2b5s4pKP77PjA0tNNPLj5+05dcKep4HByuK8vsjwmBRwAAAQASURBVDsq1ZzNcHJx1RsxORC7dYkc7kt/rc6+GyGuNlxzlInrjorum8LzX7JDyk0bbnDqRjb5sth9uOmTK51y69+Tv1Mcd7oAAHhC0gUAwBOSLgAAnjCni1Jj20VHOuWfBz0VsW3wAPkcnRPV83/c1s7jhh9AHzwxaEtot1PX/bFBJj5gtnvSDJIrtf7+Jm6QXvhSIhGRtN0V8Uyg+HtkxslOuc+xL0VoKdK+72wT/3yg/X5Gv4s/dtrdUGuRidPVrybO0eGz/JHvEYPv56zRN5q49T+Ss+tUUbjTBQDAE5IuAACeMLwcRqXZf5LK1fcksScVz+qe7jEBRW1cHhwOjmXT9fAD6IPP8X9rujt1DT/708TJ2sUGhdt2dHMTn5PxUVgt9xTx1mhEulOe3NUO6x5R2Z3mcZb4XB95uU/w3Rvt+zq4M5yIyMjxdti7xb9+MXHY27xU4LcSAABPSLoAAHhC0gUAwBPmdMOkNG9i4l+PfjFiu+CykgYf888Yq9S6dnu+Sw6dksSeWE82+NYpfz0+w8RPHdvNxLlr1gpKj5Swe4jgezRtO7Px8ZD25TSnfPvg/ib+9sFhcX2tFbnZTvmRtT1NvPyqxk5d89/t0qDSOI8bxJ0uAACekHQBAPCEcdFwGzeb8OBXbjbxYcfPdZqteLS1iTPeT/7JFWVVTnt78Ph9+0+I+/Of+vv5Jl47qaGtUG67uy6zpxZdlLnaqTux6nYTP1U58glESK7wJSavbDnYxOlfTAtvjjio96ndTapL41ucuun9h5bouc8ZeodTPvCJ4G5w80v03MnEnS4AAJ6QdAEA8ITh5TB5GzaauHlgs+wNYe2qSun4pm1Zl77aDucfO/0yp+67Lq9HfNzqvF0m7vmqPZCg1YgVTrvKq+xQceOcyBvij3mui4nfqnaUU7f10AYmztxadoe1KpqRc44xcROZmcSelF95a9eZuPF/1jl1Z/6na4me+0Apn4eLcKcLAIAnJF0AADwh6QIA4AlzukiqvIVLTFynt1t3pkQ3J9RM7Nx7bhHtiuzHn39GrKv2x3LbLsbnR2Ks7BG5LuPjjMiVQJJwpwsAgCckXQAAPGF4GUCZlbLbbi32+IaDnLo6L00Obw4kHXe6AAB4QtIFAMATki4AAJ4wpwugzGp5248mniRVk9gTIDrc6QIA4AlJFwAAT5TWOtl9AACgQuBOFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADz5f3XpyC6H9WCIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义画图模板\n",
    "plt.figure(figsize=(8,8))\n",
    "# 随机输出16个图片\n",
    "for idx in range(16):\n",
    "    plt.subplot(4, 4, idx+1)\n",
    "    plt.axis('off')\n",
    "    plt.title('[{}]'.format(mnist.train.labels[idx]))\n",
    "    plt.imshow(mnist.train.images[idx].reshape((28,28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，定义用于训练的网络，首先定义网络的输入。\n",
    "\n",
    "这里我们直接使用上面的数据作为输入，所以定义两个placeholder分别用于图像和lable数据，另外，定义一个float类型的变量用于设置学习率。\n",
    "\n",
    "为了让网络更高效的运行，多个数据会被组织成一个batch送入网络，两个placeholder的第一个维度就是batchsize，因为我们这里还没有确定batchsize，所以第一个维度留空。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 定义模型（计算图）- 三层神经元"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义输入x和输出y变量（占位符，未赋值）\n",
    "x = tf.placeholder(\"float\", [None, 784])\n",
    "y = tf.placeholder(\"int64\", [None])\n",
    "# 定义学习率（占位符，未赋值）\n",
    "learning_rate = tf.placeholder(\"float\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "# 从截断的正态分布中输出随机值，stddev是标准差\n",
    "def initialize(shape, stddev=0.1):\n",
    "  return tf.truncated_normal(shape, stddev=0.1)\n",
    "\n",
    "# 定义第一层神经元个数100个\n",
    "L1_units_count = 100\n",
    "# 第一层神经元的权重，每个神经元有784个输入\n",
    "W_1 = tf.Variable(initialize([784, L1_units_count]))\n",
    "# 第一层神经元的偏置，100个神经元就是100个偏置\n",
    "b_1 = tf.Variable(initialize([L1_units_count]))\n",
    "# 计算第一层神经元未经过激活的logits_1\n",
    "logits_1 = tf.matmul(x, W_1) + b_1\n",
    "# 对logits采用relu激活函数\n",
    "output_1 = tf.nn.relu(logits_1)\n",
    "\n",
    "# 定义第二层神经元个数10个\n",
    "L2_units_count = 10 \n",
    "# 第二层神经元的权重，第二层每个神经元的输入数量=第一层的输出数量\n",
    "W_2 = tf.Variable(initialize([L1_units_count, L2_units_count]))\n",
    "# 第二层神经元的偏置\n",
    "b_2 = tf.Variable(initialize([L2_units_count]))\n",
    "# 计算第二层神经元未经激活的logits_2=第一层的output_1 * W_2 + b_2\n",
    "logits_2 = tf.matmul(output_1, W_2) + b_2  \n",
    "# 把第二层未经激活函数输出的logits_2赋值给logits\n",
    "logits = logits_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来定义loss和用于优化网络的优化器。loss计算使用了sparse_softmax_cross_entropy_with_logits, 这样做的好处是labels可以不用手动做one_hot省了一些麻烦。这里使用了sgd优化器，学习率为可以根据需要设定。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用softmax交叉商损失，sparse_softmax_cross_entropy_with_logits可以不用对labels进行one-hot\n",
    "# tf.reduce_mean()对batch_size里每个样本的loss求平均\n",
    "cross_entropy_loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "# 使用sgd优化器\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy_loss)\n",
    "\n",
    "# Adam优化器\n",
    "# optimizer=tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要注意的是，上面的网络，最后输出的是未经softmax的原始logits，而不是概率分布，要想看到概率分布，还需要做一下softmax。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Softmax_1:0\", shape=(?, 10), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# 将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。\n",
    "# 对logits做softmax得到输出概率分布\n",
    "pred = tf.nn.softmax(logits)\n",
    "\n",
    "# 评估模型性能：对比模型的输出和labels有多少能匹配的上就行，如果y==pred则True，否则False\n",
    "# tf.argmax(input, axis=None, name=None, dimension=None)： \n",
    "# 此函数是对矩阵按行或列计算最大值，axis：0表示按列，1表示按行\n",
    "# 返回：Tensor  一般是行或列的最大值下标向量\n",
    "correct_pred = tf.equal(tf.argmax(pred, 1), y)\n",
    "\n",
    "# 获取准确率：因为是按batch进行模型训练，所以要对batch做均值处理。\n",
    "# 因为correct_prediction是Boolean类型，所以把它强制转换成float32类型\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 一个批次取32个数据\n",
    "batch_size = 32\n",
    "\n",
    "# 训练次数：1000次\n",
    "trainig_step = 1000\n",
    "\n",
    "# saver用于保存或恢复训练的模型。\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上定义的所有操作，均为计算图，也就是仅仅是定义了网络的结构，实际需要运行的话，还需要创建一个session，并将数据填入网络中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 100 training steps, the loss is 0.207517, the validation accuracy is 0.885\n",
      "after 200 training steps, the loss is 0.627947, the validation accuracy is 0.8824\n",
      "after 300 training steps, the loss is 0.454078, the validation accuracy is 0.918\n",
      "after 400 training steps, the loss is 0.141378, the validation accuracy is 0.9068\n",
      "after 500 training steps, the loss is 0.171617, the validation accuracy is 0.9306\n",
      "after 600 training steps, the loss is 0.119474, the validation accuracy is 0.9324\n",
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/training/saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "after 700 training steps, the loss is 0.344888, the validation accuracy is 0.948\n",
      "after 800 training steps, the loss is 0.35895, the validation accuracy is 0.9408\n",
      "after 900 training steps, the loss is 0.106414, the validation accuracy is 0.951\n",
      "the training is finish!\n",
      "the test accuarcy is: 0.9509\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    # 初始化模型里的变量\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    # 定义验证集与测试集的变量值\n",
    "    validate_data = {\n",
    "        x: mnist.validation.images,\n",
    "        y: mnist.validation.labels,\n",
    "    }\n",
    "    test_data = {\n",
    "        x: mnist.test.images, \n",
    "        y: mnist.test.labels\n",
    "    }\n",
    "\n",
    "    for i in range(trainig_step):\n",
    "        # 从mnist的训练集里获取一个批次进行训练\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        \n",
    "        # 运行optimizer和cross_entropy_loss，并且给模型的placeholder图进行赋值\n",
    "        # 返回值：_ 表示的是optimizer的返回值，但是我们不关心这个返回值，所以用下划线接收\n",
    "        #     loss 表示的是cross_entropy_loss的返回值\n",
    "        _, loss = sess.run(\n",
    "            [optimizer, cross_entropy_loss],\n",
    "            feed_dict={\n",
    "                x: xs,\n",
    "                y: ys,\n",
    "                learning_rate: 0.3\n",
    "            })\n",
    "\n",
    "        # 每100次训练打印一次损失值与验证准确率\n",
    "        if i > 0 and i % 100 == 0:\n",
    "            # 在验证集上的准确率，调用run方法前，需要对之前的placeholder变量进行赋值\n",
    "            validate_accuracy = sess.run(accuracy, feed_dict=validate_data)\n",
    "            print(\n",
    "                \"after %d training steps, the loss is %g, the validation accuracy is %g\"\n",
    "                % (i, loss, validate_accuracy))\n",
    "            saver.save(sess, './model.ckpt', global_step=i)\n",
    "\n",
    "    print(\"the training is finish!\")\n",
    "    # 最终的测试准确率\n",
    "    acc = sess.run(accuracy, feed_dict=test_data)\n",
    "    print(\"the test accuarcy is:\", acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 测试模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用我们训练的模型做一个测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model.ckpt-900\n",
      "0.9375\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHiCAYAAACtERYWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XeYFMXWBvD3bGAJS86SlpxFRTGgAuaEOSOi13ANfBiuqFcxZ+81YU6YMyiiXjGjIkoSRMkSJErOYWF3z/dH1XZ3DTu7szszven9PQ8Pp6dqqqu3p7u6qzqIqoKIiIiSL6W0K0BERFRZsNElIiIKCRtdIiKikLDRJSIiCgkbXSIiopCw0SUiIgoJG10iIqKQlPtGV0TuFJHdIrJVRGrE+J0FIrJLRN4sJI+KyDYRuS9xtU08Ecmwy75bRO4t7frEi+uzYq1PgOsUiG15ygsRedUuy+IY83ew6z5XRC6NkqeviOTZfMcltMIJJiJH2XrmichRxf1+mWh07QIE/+WKyJPFKOI9Vc1U1W22vM8jytslIr/nZ1bVtgDuj6HcHqp6a6Ce/UXkD1vmBBHpEkjLEJHHRGSFiGwQkWdEJD3K8jYQkZ9EZJ2IbBSRn0WkdyD9SBFZJCJ/i8i5gc/riMivIlIzsCzZqpoJ4K0Y/1ZJZf8OL4vIXyKyRUSmi8jxxSwmcn0OtX/3LfbvMjSYubTXp82fKiL32vxbRGSaiNSxaeV2feYTkcEiMkVEskXk1RIUEblO+4nIdyKyqaCddxlZp0fY9bNZRBaKyOWBtB4iMlNE1orI9YHP00Vkooi0KOHyhEJE6onIR/ag5S8ROb+YRTysqlmB8jJEZIT9W/0d/Juo6jz7m/6xiDJX2N/IWFumiMitIrLElvuuiNQKzLOZiHwsIutFZJmIXFHI8hZV1lC7LmeKSPfA571FZHSwLFX92i7PkqL/THsqE42u/UNn2gVpAmAHgA/iKO/4iDInxFMeAIhIe5gd4RUA6gD4BMAYEUmzWW4GsD+AbgA6ANgPwLAoxW0F8A8ADQHUBfAQgE8CZT0OoD+AYwE8IyKp9vMHADyoqlviWZYkSwOwFEAfALVh/gbvi0hWHGUKgAth/lbHARgcbLxKVGBi1ycA3AXgEAAHA6gFYCCAnTatPK/PfCsA3AtgRILK22bLGlpUxlglcp3axvgjAM/D/I7PAfCoiPSwWR4AcAOAHgBuFZEm9vPrAYxS1aWJWq4keRrALgCNAQwA8KyIdI2jvDsBtAfQCkA/ADdK/GesF8JsR70B7AWgGoDgydibABbBLMOJAO4XkX7FLUtEmgK4BEAbAM/CrFvY380jAK6NczkcZaLRjXAGgNUo+qgoJnZnfxiA1+Ms6lgAP6rqeFXNgWkom8E0LoDZqQ5X1fWqugbAcJiGdQ+qulNV56pqHkyDkgvToNSzWWqo6h+q+hvMhlFfRHoBaK2q78e5HEmlqttU9U5VXayqear6KcyG0TOOMh9W1V9VNUdV5wL4GGbjiUfC1qeI1IXZMC9T1b/U+ENV8xvdcrs+86nqh6o6GsC6BJU3SVXfALAwEeVZCVunMNtiLQBv2PU5GcBsAPlnzq0BfKuqywHMB9BSRFrB7L8eS+AyJZyYLv4zANymqltVdTyAMTCNUkkNAnCPqm5Q1dkAXgRwUZxV7Q/gZVVdqqpbYdbnOSJSXUQyAfQFcJ+q7rbb1khEX59RywLQEsA0Vd0M4GuYxhcw2/QYVV0c53I4ymKjOwjA6xp4KLSYLthDS1jehTAb4uIE1E0iYoE5ao6W3lxEakctTGQGzNnQGAAvqepqm7Tadl/1AJAHYAOAJwAMiX8RwiUijWHOKmYGPivx+hQRgTmImllU3liKi4hLuj67A8gBcKbtWpsnIlcH0ivM+owmzm00kRKyTlV1FYB3AFwsZujgYJizuPE2yx8AjhGR5gCyACyAWadDVXV3gpYlWToAyFHVeYHPfgPQFQBEpKVdny1jKcwedDa1ZexRXpwi11cGzBm1REkPrutYy/oTQHcxw0FHAZhphwfOBfDfuGpfgDLV6NojxT4AXgt+rqp17NFYSVwI4NU4qwaYI6A+Ygb8qwC4BUAVANVt+lgA14hIQ9vVlL9Drb5nUYaq7g1zNH0+/I0ZMN1jTwB4Aebo80o7/6oi8oWYsbA+keWVNbaL7i0Ar6nqnPzP41yfd8L8bl+Js3qJXJ/NYbogO8CcAZ0J4E4ROdqmV4j1WZg412miJHobfQfA7QCyYXrebg10G98Asx7HALgOpudlC4BFdpzxexE5K7GLlzCZADZHfLYJQE0AUNUldn3GOmaZGShjj/LiMBbApSKSZQ+MbrKfV7dDMj8BuE1EqorIfjBn79HWZWFlrQNwH4BvYbqpb4DZXm8CcJpdlx/bA6y4pRWdJVQDAYxX1UWJKMweeTeB6XYoLN/nMGdPAPBPVd3jIhZVnSMigwA8BXNU9yaAWQCW2Sz3wYwjTYfZSF8EsC+AVYXN23ZBviMis0Vkuqr+pqrTYbpO8scbHoEZK/wepstjBYAfRKRVsEegLBGRFABvwHSnDk5QmYNhDqIOU9XsQvKFvT532P/vVtUdAGaIyLsATgDwVUVYn6Ut7HUqIp0AvAvgdABfwZwRfSoiK1T1M1X9C2b9wnZR/gzgGJhxwvcAfAbgDxH5RlXXx/8XSKitMAf7QbVgDhpKWl5+GTsDcdTyRGRrYLJLlGwjALQAMA6mrXoEpps4f30OgBmbXgozTPEmop9dF1qWqr4Dc5AFETkR5vcxDf4Z+8kwZ71xXUsClLEzXZgd6mtF5ordIAAf2j78qCIuvIp61aiqjlTVbqpaH8AdMN1Kk23aDlUdrKrNVLUNzNjXVDtuG4t0+GMJQY8BGGZ35t0BTLFd5ekwF2KVObYL+GWYCxzOSER3m4j8A+ZCmCNVdVlheUthfc7ILzJYfJTZlrv1WRaUwjrtBmCeqn6h5tqEuTANaUFX4t8O4EXbJZ2/TjfB7NDblXypk2YegDR74Vm+HijhkI2qbgCw0pYRU3mBdZkZ7Yza/t3vUNUsVW1uy1tu/0HN9RMnqWpDVT0QQAMAk0pSVj4RqQZzlfm/YA60ltqx3skA9i76r1G0MtPoisghMBc9xHWVcaC8agDORmK6lvPL7GnHdxrCdBWOye82FXP5+l5iHATgNpiNvqByDhKRQ0WkiohUE5GbYBqoiRH5jgZQVc3FSIC5IOkIe5VhBhJ0UUsSPAugM4D+tnGJi4gMgNkQjlbVhF14k6j1qaoLYLsfxdw60RnmiPjTYL5yvD4hImkiUhVAKoBU26VX4p4yEUmx5aWbSalqu4TjrWdC1inMWU57MbcNiYi0BXAS/AOs/Pl1genFeNZ+lL9OG8PstEt0W0kyqblt60MAd4tIDTG3K54C0zNVUq8DGCYidW0vwWWIc98r5ramtvbv3wXAozC9SXk2vbOI1LT70QtgehoeLUlZAcMAvKqqK2DWXUe7LvshURf9qWqZ+Adzaf4bUdK2wnQpFpR2J4A3C/j8PAB/AZDifC+QrgDaRXw2HqbLZL2tb41A2uEAFgPYDmAugAER3/0cwC027gPTbZFf1vcADo/InwHTDdYq8NmRdh4rAZwbkf9VAPeWgfXYyv7tdtr1lv9vQCBPsdYnzI5sd0R5z5WV9Wmnm8GMG22F2Tj/WRHWZ8TfVyP+3RnHOu1bQHnjytg6PRvmgqktMGetDwFIifjOdwAODEz3gOnSXgvg+uIsT8jrsx6A0TC3bi0BcH4graVdny2jfHeP36b9fY+AGSteFbnsNs84AJdGKbMvgGURn3Ww62k7zL488u95LYA1dhnGA9g/It37TRZVls3TCeaMNjXw2VC7LmcB6B6RfzGAo4r9ty/tlZ+AH88w+0ffGNzAivjOXLtCRhSSZyfMxQD3lPYyFrEsGXbZtwG4o7Trw/XJ9cl1WrLlKS//YMbCtwJYEGP+9nbdbwdwUZQ8h8NcG7ERwLGlvYxFLM+Rtp47APQr7vfFFkJERERJVmbGdImIiCo6NrpEREQhYaNLREQUklAfjnF0ylkcQC4lX+V9IEXnKh6uz9KTjPUJcJ2WJm6jFUu09ckzXSIiopCw0SUiIgoJG10iIqKQsNElIiIKCRtdIiKikLDRJSIiCgkbXSIiopCw0SUiIgpJqA/HIEqGxfce7MW5Vd1nATTsusaLf+4xKmoZbb+92ItrTqrmpDUePiHeKhIRAeCZLhERUWjY6BIREYWEjS4REVFIOKZL5dKGz9p78R/7PBXTd3YX8uj3Of1e8uK39m/qpL3/VR8vzp09P8YaUlkiPbs605+NecOLuz832Itb3MPx+7Cl1qntxXOfauPFwW0SAIat7unFvw/o4KTlzpqXpNolHs90iYiIQsJGl4iIKCTsXqZyIdidDAA/7fNuTN97bqPfXfXoz0d7cVarNU6+L7t86MUDaq500u67qIEXt7mJ3cvl0eoDajnTOcj14uor+MrZ0pTXurkX/973eS+OHA66t9FUL+5x2iFOWgt2LxMREVEkNrpEREQhYfcylVk5R/pXK37b4+mI1HQvenyDfyXjd+fs72ZbsdoLO2yY4sUpVas62e6f2N2Lb2nwu1uPujkx15nKpg175zrTy3Kyvbj+yz+HXZ1KLa1Fc2e69Qt/llJNSgfPdImIiELCRpeIiCgkbHSJiIhCUq7HdNdddrAz3XKgPzYwZ3VjJ21Xtj8G2OwdP66+bKuTL2/6rERWkeKwtVkVL06JOD4MjuOOO9kfj81dODemsv+8a19n+u16jwSmMpy05mN5bFoeae99vPjHkx510vr88H9e3A7TQqtTZbXkdv8Wn57HufvYh5v+WOzyMg9xb/lbeptffoMZ/jUY1T6eVOyyk417EyIiopCw0SUiIgpJue5evnHo2870GTU2+BNtC/liXz9cnLPdSXpiTb/4KxajSatbeXGNR2o7aWnfTI3MXunUed2/lePMKRc4abJhsxfnrFxc7LIvPeFrZzozJSNKTiqv1nep5sVNU6s7ac1GpkdmpySa8c8nvXi35haSMzbjerzlftDDDz/a5r+wZMSWU51sad+W/n6VZ7pEREQhYaNLREQUEja6REREISnXY7rDbznXmb59b/8You5s9xUVGzqLF1fZe6MXP9ztQyffY00nevFn2zO9+MTq7q1Fhdmhu7x4YnYNL+5bdbebMTCvduf800nq8E3Ms6sUEvGS6sX3+beYXVLnvxGp/mMh/7XyICel5tez/XrEXQsKy5FX+dcEjN5Wx0nLHOffWsZ1mhzp4/yx1XRJjbu8abvyvHjx7oZO2mk11nvx2Zn+o1/PfuMFJ99JzXqitPFMl4iIKCRsdImIiEJSrruXa4ycGDEdPW+tKJ8/2aSvM31v7yz/O9/7T7h6uG+7mOuVtsPvBqkxw38hev0fRjn5ulcJPBlrMW9hSIaNA/0u5Z8u9LuUa6e4bxn6Odvv/pp+r/u0qmqby95TbWhPqV07OtP3N3rHi1/e7L7ZJnfjplDqVJnsOLWXM31x0w+8OHibUKy3DHX75gpnuuE3/m19GZvcMv7d1z9//P2s4VHLXPZv/8lVzR+YEFM9Eo1nukRERCFho0tERBSSct29nAg5f69ypmuM8qeDHRg1Rq4rUfmrLvW7N7tWcf/c/13vd4dlvbLQrVeJ5kaR1u7nX8Ue2aUcNGjcpV7cYTS7k8uj5UfXj5o2dUuriE92JLcylUSwS//eR90rhfevsiuYM2oZwSdIDfvuDC/ufOMcJ1/u5s2IpuN8/wUok072t/NeGTudfJ9f+bAXH1P1Rict637/aVWanR11XvHimS4REVFI2OgSERGFhI0uERFRSCr9mG4ypLVq4cVP3fKUF0c+leWDJ47y4vorfwbFb9dX7tjdz52CL6f3x3p6/DzIydf5Xwu8mE8oKp82d9kdNW36U/s403XA7S0R8gLXqbhjuNH946/jnOkt5/hvg+qwzL+eojjbYfCJdVe96t9qNOWfjzv5mqb68/r1EjftjA/9fYL+NhvJwjNdIiKikLDRJSIiCgm7l5NgznXNvPiADP9FCzN3ubcp1Ju1PbQ6VWRpbbK8+J52HzhpdQO3CU0N3AXQ6h638yp3w4ak1I2SK/v4A7z442OedNLuXus/3L7eqBlOWh4oTLes2t+LN1/q3tqVu2x+QueVNWqtF992qvvykgebTE7ovEqCZ7pEREQhYaNLREQUEnYvJ0D2iQc407+e+Vhgyn9I95XXXOPkqzaBTz5KhLbvL/fifatEP448L/AA9Q6/lX43E8Vv2RH+LmzvKu4TxwYt7u7Fjba5TzeixCvsnbkz9gu+3zyx3cl7EH9ILy3FHUgorI4r7vLjJqcmvFYenukSERGFhI0uERFRSNjoEhERhYRjugmw5Hj32CVT/HHc8xYd7cXVx/7m5FNQSW0Y5L+96a7GwadOZTj5Bi32n/rV+cY/vZhPnaoYGnZb7cW56o7fpX1cN+zqVDpzr6zuxbG+nD7ZFp/u35I0sqF73cxuTQ3Ebn33usOPk3lLGc90iYiIQsJGl4iIKCTsXi6hlJo1vXjgYeOdtM15/ouTV9/fxoszsnmbSkmlNdvLmT5syEQvzkzJiMzu+XlWOy/usIF//4ogrbX/Uov/dvSfQPbiphZOvnoj+FKDZBt22CelMt+0Fs2d6S09/f3Dcxc/E1MZk7LdW8xkV078FYsBz3SJiIhCwkaXiIgoJGx0iYiIQsIx3RKaf2dXL/60gTuGcMr8M7w4438cR0yE2be443WjmxQ8ltTv97Ocad4mVPHM/6c/fndQYDj/sl/7Ofla4I+wqkQhm3VXE2d65jFPxfS9UVsbePGzN7j7iqqzw3ksL890iYiIQsJGl4iIKCTsXo7RpgvclyHPOGe4Fy/I2e2kbX3Iv5w9AyuTW7FKYurJj0V8UvBtQrWvcp8lk8OX01c4eS12Fvj5jo1VC/ycKob0cU29+IGmo0pUxqvLD/Hiqp+UzlveeKZLREQUEja6REREIWH3ciGCT0G69rb3nLQM8f905/420Elr+DmvWC4tuxvXdqbTdzUrdhm5a9Y605qd7cWS4XdrpzZsgGhyG9Zxpuf/q0pM89Zc/wXcnf7vTyctd/PmmMqo6J458M0CP2/2efQXlFNypIo/nFPYC+I3n39Q1LS77n7Zi/tVK3joILL8PV+uENu61yOWx5QvmXimS0REFBI2ukRERCFho0tERBQSjulGkDT/T9Lj02VefFbmOiffW1saeXHj29xjl2S+AJkK99nIEXGXcci085zptatqeXHdhlu8eGLPt+OeV2G6DBvsTLe5sXK+NWdn/17O9KFVg7d6cBdWmh5870wvPvuSx6Pm++E/T3txYS+7362xzbewMoK6fXOFM90ev8Y2gyTimS4REVFI2OgSERGFhH0zkXp09MJ7Gr0RNdvT9/sPy67zW+Xs9gvTKbMGONPfdBuZtHlN2PedEn1vu+7y4t0afZDhhBkXefGm6dFvO2o2PpyXapd1S052+xyDt+vdvba7F2d+PNXJF2NPJcWhzXv+7XWTLnCfCNYrI/rtP/GKfAH9C3/38eINV/kvQ+i0KOK2u6TVKHY80yUiIgoJG10iIqKQsNElIiIKSaUf003t0sGZvvzdjwvM12XE1c501hu/JK1OtKdqxy5yprve799OozH+imt2Wu/Fxbndp+uPF/vzWlIjar42I7f6E5N+j5qvLuYXGJMvtZZ/m9ZNvf8XNd/bnx/uxW1yeG1F2HJnzfPi26+/1Elb2t+/rmHe8c8ndL5XjXBvBWpx34TAVNl+sxjPdImIiELCRpeIiCgklb57ec5VdZ3p/tULfpNL83G73A+UNySUpta3xNeVeBJ6xj4vzIhrXlR8eYE3O83avpeTdtTy/b24/f0zvbgs3A5SmVX72H0pfIfASN3h5/nDc+kXrXLyje3qv8HtmD/O9eK8Vxs5+dR/AReypq9x0srTuueZLhERUUjY6BIREYWkUnYvBx+g/k3/RyJSq4dbGSLagwa6l+fu76ZVwV9eXJ66FSuzWu8E7vaIeODbafD3xzWwMJCyENGU5/XOM10iIqKQsNElIiIKCRtdIiKikFTKMd0VvVO9uGVa9DHc4Ivq0ze7twzxhiEiIiounukSERGFhI0uERFRSCpl93JhHljXxYt/PjbLi3Vl9AfYExERxYJnukRERCFho0tERBQSNrpEREQhqZRjum1u9t9Qc8LN+xWS8+/kV4aIiCoNnukSERGFhI0uERFRSET5MnYiIqJQ8EyXiIgoJGx0iYiIQsJGl4iIKCRsdImIiEJS7htdEXlVRHaJyOIY83cQka0ikisil0bJ01dE8my+4xJa4QQTkaNsPfNE5KjSrk+8ROROEdltl6lGjN9ZYH8DbxaSR0Vkm4jcl7jaJp6IZNhl3y0i95Z2fRKB26hcYuupItKutOsTL26j8W2jZarRFZH2IrKzsBUTxcOqmlVAefVEZI2IjM//TFXnqWomgB+LKHOFqmaq6lhblojIrSKyREQ2i8i7IlIrMK9mIvKxiKwXkWUickUhy3mLXWn5/3bYHUgDmz5URNaKyEwR6R74Xm8RGR0sS1W/tsuzpIjlCY2IDBaRKSKSLSKvlqCI9+zffpstr5+IfCcimwracatqWwD3x1BuD1W9NVDP/iLyh10HE0SkSyAtQ0QeE5EVIrJBRJ4RkfRoBYtIqojca/NvEZFpIlLHph0pIotE5G8ROTfwnToi8quI1AwsS7Zdn2/FsDyhK8/baGHzLCDPuSIy1/7mVovIaxHb++P2d/GziDQPfH6+iAwPlqWqL9vlKTNEpLOIfGuX708ROa2YRURuo3Xs32i1/XdnMHMc2+gRdhvZLCILReTyQFqh+9ECljnL7ke2i8gcCZykhLmNlqlGF8DTACYnsLyHAMxOUFkXAhgIoDeAvQBUA/BkIP1NAIsANAZwIoD7RaRfQQWp6v32B5tpV95DAMap6loRaQrgEgBtADwL4AEAEJE0AI8AuDZBy5NMKwDcC2BEgsrbZssamqDyICLtYTaaKwDUAfAJgDH27wwANwPYH0A3AB0A7AdgWCFF3gXgEAAHA6gF81vZadMeB9AfwLEAnhGRVPv5AwAeVNUtCVqsMJTnbbQ48/wJQG9VrQ2zLabB/KYhIr0A9ATQBMB4mN8KRKQ2zG+0sN9JqbO/8Y8BfAqgHoDLAbwpIh3iKPYxANUBZAHoBWCgiFwcZz3TAXwE4HkAtQGcA+BREekBFL4fjVLkOwCmAagP4FYAI0WkoU0LbRstM42uPbrYCOCbBJV3CMwO85VElAezQl5W1aWquhVmBZ8jItVFJBNAXwD3qepuVf0NwEgA/4ihngKzs3jNftQSwDRV3Qzga5gNHjCN7RhVXZyg5UkaVf1QVUcDWJeg8iap6hsAFiaiPOtYAD+q6nhVzYFZn80A9LHp/QEMV9X1qroGwHBEWZ8iUhdm/Vymqn+p8Yeq5je6Nez0bwB2Aahvd9ytVfX9BC5TUpXnbbS487RlBHfeuQDyu4ZbAxivqtkwf4v8bfQ+AP+x225Z1gnmoOQxVc1V1W9hDjIGxlFmf5jejO12H/UyYtj/FaEezAHsG3abmgxzsNQlMmMB+9HI9PwD5ztUdYeqjgLwO4AzbJbQttEy0ejabpu7AVxfQFpLEdkoIi2LUV4qgKcADAaQyKd/SEScAaB94PPI9G4xlHkYgEYARtnpPwF0t12TRwGYKSItAJwL4L8lr3rZYdfnoaVdD+y5viLXWWR6c3s2E6k7gBwAZ9ruqXkicnUgfbWI9LBH6HkANgB4AsCQRCxEGCrANlrseYrIoSKyCcAWmJ3z4zZpJoDDRKQagCNhttH9AXRU1bcTtSAhc377JdxGS7L/i0pVV8GcnV4sZvjmYACtYHoXIkXuRyN1BbAw4oz1N/s5EOI2WiYaXQD3wByhLotMUNUlqlpHVYszZjkEwERVnZqwGgJjAVxqxwVqA7jJfl7drsifANwmIlVFZD+YjbR6lLKCBgEYaY/MoarrYI6Yv4Xppr4BZuXfBOA0EflezNhx82gFlnV2fUYdTwvJ1wD6iLkgpwqAWwBUgb/OxgK4RkQaikgT+BtfQeu0OUz3VweYs6AzAdwpIkfb9Ctg1uELMGcTV9r5VxWRL+w4U589iy1TyvU2WpJ52l6Q2jDr9z8AFtvP/4DZuf8C0zP1MExPyBARGSIiP4jIW/bAuSyaC2A1gKEiki4ix8D08Hi/7RJso2MB3CwiNcVcLPYPxLb/K8o7AG4HkA0zxn+rqi4tIJ+zHy1AJoBNEZ9tApA/VhvaNlrqbxkSkX1gzuj2TVB5e8FsXD2L8Z3gitqj68IaAaAFgHEwf7dHYLpU8ndCA2DGu5bCdIO+Cf8oKtp8qwM4C8Apwc9V9R2YHxtE5ESYH9w0+EdmJ8Oc9Z4L2oOIfA5z5AsA/1TVPS54UNU5IjII5synKcz6mgV/fd4HM9Y7Hebv/yLMb3RVAbPcYf+/W1V3AJghIu8COAHAV6o6HWb4AXbM/hGYsd/vYbqlVwD4QURaaRl8LmtF2EZLMs98qrpcRMYCeBemixKq+hjMOCZsr8YPMCcxl8P8nW6CGeu9ubjzSzZV3S0ip8KMd98EYAqA92F+5yU1xJY3H2ZY6R0A50XLHMs2KiKdYP7mpwP4CqbH4lMRWaGqnwXyFbgfjbAVpqs6qBZMLwbC3EZLvdGFWdAsAEtMtzwyAaSKSBdVLey9e9H0gtmJzrLlVQNQTUT+BtBMVXMjvxB5ZaGItCkgTx6AO+w/2KPD5fYfVPUvACcFyngbwKQi6noagPUwO4k92O6r+wEcD/ODW6qqm0VkMsyZGRVAVY+PMd9ImLF32LOSS2AvErKN52D7D2KumpxqfweRZuQXGSw+ymwfAzBMVXeIuTJ9iqrusheNNIQ5Aylr+qL8b6MnF3eeEdIAtI38UEQawzS0B8M08DNsozYZwDVFlFlqVHUG/OsXICITEGU8NMby1sOceOSXdz8K2f/FuI12AzBPVb+w03NF5DOY/eFngXyF7ketmQDaiEjNQBdzDwAFDQckdRstC43uCzBHM/k3lts7AAAgAElEQVRugNnAryxheZ/b7+c7B8D5AE6JYcOKSkTqAagLcxbbGcCjMGc2eTa9M8xZUjaAswEcY/MVZhCA1ws5choG4FVVXSEiCqCj3cj7IbEXFSWUmKsj0wCkwuycqwLIsRcslaS8FJiu33QzKVUB5Knqrjjr2RPmTLYeTC/FGFWdY9OawTScKwEcCOA2mEZ5D6q6QER+BHCriAyBubDmXEQc6dvu5qqq+qn9aBGAI0RkKczYY0IuPEuCcr+N2jOrmOcpIgNgLrRbIiKtYHo+CrqA7FEAd6rqdhFZBOAA8S+sLMvb6N4A5sGcnV8Fc0DyahzltYW5yG4jzL7vcgQa9RKaBqC9iBwB4DuY7eokmO78oKL2o1DVeSIyHcAdIjIMpuHeG/6FVPnLkfRttNTHdO3Vbn/n/4PpBthprxjNv0hja6wXaai5hypY3iYAu20cjwYA/gdz+8rnAEao6guB9GNhNrINMOMDx+Uvg12OrSJyWGC6GYAjALxe0Mxs18oxMGNFUNWVAB6EOWIbAuDfcS5PMg2D6XK9GcAFNvZuo4j8W8TgcFvG/2DG0HYA+DIB9XwCZicxF2a9XRZIawtgAsz6fg3AzarqzVNEPheRYG/DeTAXeayDOQq/TVW/CeTPgBkXDJ79/B+A52DGjq6Kp8FJpoqwjRY1zwKWoQuACSKyDeZ6jblwfx+wjUEdVf3IzmMSzLpfCnNg/GCcy5NMA2EOKFfDXAx2tJqrsQGUaBvtCXM18BaY22wGqOrMeCqoqgtgxoaHA9gM09U7CsBLgXpG3Y+KyHMi8lzgo3NhbgPcALNuzozYR4ezjapquf4HM9a2FcCCGPO3h9nRbgdwUZQ8+Tv5jQCOLe1lLGJ5jrT13AGgX2nXJwHLMwxmp7kR5jL+WL4z1/4GRhSSZyfMjvae0l7GIpYlwy77NpjbG0q9TglYpsq+jV5s67kTQJvSrk8ClofbaBzbKN+nS0REFJJS714mIiKqLNjoEhERhSTUq5ePTjmLfdml5Ku8D6ToXMXD9Vl6krE+Aa7T0sRttGKJtj55pktERBQSNrpEREQhYaNLREQUEja6REREIWGjS0REFBI2ukRERCFho0tERBQSNrpEREQhYaNLREQUEja6REREIWGjS0REFBI2ukRERCFho0tERBQSNrpEREQhYaNLREQUEja6REREIQn1JfZlRW6//bx48AvvO2nPtm+XtPluOecgZ7rO9LV+neb+mbT5UvFsvPBgZ3rig896cZenr/Lilg9NcvJpTk5yK1bBpbVq4cWN3tvoxd9P7eLk6/SMn5Y7c27yK2alNmzoTK873t9X1H3vVy/W7OzQ6kTlD890iYiIQsJGl4iIKCSVsnv5r2MzvLhe6tbQ5vv3ibuc6d0D/WOeeieFVg0qQFqzvbz4nttfippv1tXPePHxww9z0nTLlsRXrAJLa9LYmb573Cgv7pie58VHrGvi5MudOT+5FQsIdikPGP+rk3ZQ1Y+8+Orf/+knTJuZ9HqVZ6kN6jvTcx9r6cV92/vrdnmf3U6+itJtzzNdIiKikLDRJSIiCgkbXSIiopBUmjFdSa/ixUccMb1U6lBzWlVn+uxLvvfi7+o0d9JyN24KpU5krD62lRcfU3131Hz7TTnHixtunZfUOlVEac2beXHt97Y7aXtXSfXijl9f4cXtB7ljqWGafW+WF5+dOdZJ2+/xG714r2kTwqpSubR68CFefMc1rztpJ1b/ssDvnNqgvzOds3xF4itWCnimS0REFBI2ukRERCGpNN3LW07zn0I1vNmTXtx59GAnX3tMTFodsuuqMz2k7hwvHlezs5uZ3ctJlVK9ujN97JDxMX0v4926/oRq9IxUoA29/adOjc56Omq+zsNWe3GYz/nSg3s403+e9LwX9/n9LCetxQh/+81NbrXKpdQObb34pX897sX7VHGbnTwUbOWzNZ3ppv/0bx3LWfl3/BUsJTzTJSIiCgkbXSIiopCw0SUiIgpJhR3T1d77ONNPP/SEF7+52b89pNMw97aPZI7NHHzMH0ksnYoj+xB3DP3eRi9Hzbs9z398Z623f0lanSqi4JuDAGDNKTuj5t3/v//nxU2WhncLTnAcd9hbr0XNt/Uz93GUNdYtTFqdKoLZN/vXPwRvB4vVxJ5vO9Pzfva3w9PfuN5Ja3PfNC/O2xn9N1YW8EyXiIgoJGx0iYiIQlJhu5c3/Nt92k3zNP/Gg+v/70QvTt8wNan1SGvqd0m90tJ9os1u5TFPaVl0euzdXWfOPzUwVTGeihOWpU9kOtPze73qxcNWu0NAzV7x384T5i04y/vW8OLeGe4NLN0mDPLilk/yqVOFSe3SwZn++sjHA1PVvOihde7QzpSN/luG3mvr7iODOgSeKvjigGedtIdGnOLFeYv+iqm+pYV7fSIiopCw0SUiIgpJhepeXnfZwV78Qff/OGmvb9rbi9O/Tm6XctCsu/2rN3er22k2aPFRXpy7ek1odSLgxAN+i5q2KW+HM737Tv9l6ynsXi4WVXGmg9vAxHVZTlrqjtVIlpSa7tON5t7XxYtHn/yoF+ch3cnX8qzfk1animZtL/fl9Flp/lPfLl96uBcvO2irky+lhj8U2PMK/wr2Gy5738k3oKb/+zjcfXcMPhm1xItnnVi2n1zFM10iIqKQsNElIiIKCRtdIiKikFSoMd2UU9d68V5pGU7ay28f58XNkdxL/1O7dvTiN4/031KSre7L0Zc86l9iXyM7eW83IiP7hAO8+KlmL0bNtyzitTYp308rOCPF5X+dRjvTl4zr58VLtjT14l0vu0+CitXfh/lvgTrhwOlO2pi9nglM+eO4vaef6+Sri/klmndllOvucpEH/+8/4/nuXlwPP7v5tm3z4qaP+Pvm9/sf4OQ7r+an/oS6t3atyvbH7HVnduyVLgU80yUiIgoJG10iIqKQlOvu5dSGDZ3pYR0+i5q3+f3hPU1mzlV1vHj/DP8Wiac3dHHy1RjFLuUwrTogvehMAPp/eq0z3R5cTyXV6MlqzvR3L/j3evSr5j6Y/uWW33lxCvxbjfIeVZSEUwail/HOFv+WsPq3xPaCddpTzTNWRk3bdKzfhVzvldjKu73VmIhPop8j/jitkxd32DApthmUEp7pEhERhYSNLhERUUjKdfeyVHcfS3Js9U1e3GvyhU5aE8wOpU4A0CBrfYGfv7Vofzcf5hWYj5Kjyr4boqbN3uU/FafT8LVOWpgP369o0r51n/72xKFHePE9h2Q5acuO8buA/+z/nBdPynafanXBl1fENO/2r/tXsX72wYio+R6edawXN/ttZtR8VLgto5q6H3T1w4u6+EM0PxzQy8m2Zl//pRh6kr/v7JbudhPP3u3f/dE18PIDAPjo+Ce9+KaDLvMTfplRdMVDxjNdIiKikLDRJSIiCgkbXSIiopCU6zHdvPUbnel71uznxee3neKk/dC0rRcn+s0Taa1aONM/7fNuYMo/rtnxS4OIb3JMN9l2nuSPH005IPjia/cl9nN3N/Li3HkLkl2tSivn71VeXP3DVU5ahw/9+IQr9kM0HRDbLSEpe/u3kQRvHwKAe9d28+JW1/jXgkQ8jIyKocmYRc70vH/v8uKh9Wd58U2j3etrot3Odc6CE53pHUP8W0RPe2eck3ZxraVevGCIv89t+0sRlS4FPNMlIiIKCRtdIiKikJTv7uUtW5zpL5f73Uk/7vO2k7by09p+2vMHo7g2dnG7QDKz/C6pg/Za7NYrynNspGQP1qE47GjgdyOnS2rUfDdOPd2LW6Ps3WZAxbfkDn99R3Zhfnmf/1L1zKVlsA+yHIoctrt8qP9kt1f++6gXd0iv4X4x8PKCdl/6t/t0GjzHyZa3ze+ifvDb/k7aJaf6Q0cP7e+PU7zUw+2izvstvFtHo+GZLhERUUjY6BIREYWEjS4REVFIyvWYbqS6d/mPhexz53lO2kfdXvXih+5wX6IciynZ7nhgbuB4Zf8quyJyCwrS8snfnWm+wST5sk/dWODnwcc+AkDzl2J7AxGVXWsvd6/VmHHQ0168OGeHk1ZtTeQ2S4mW+YH/6MeLcb0Xrz/b3fZ2bsrw4s5D/dv1cgMvt4/U8eZZzvSR7f1rMr7qOsqL77jDPa9sdjpKHc90iYiIQsJGl4iIKCQVqnsZk/zu29onuEkD+w7x4o3tM1Bc9V+M3iW9/MOuzvTUA18tMF/kLU6UeKkd2jrTUw54M5jqRZ9v7ebkS//afRsOlT/bj94aNe3M6Zc6042++zXZ1aGAYFdz5gfR88X6Rq/IfenmjwLbc2B3/NDeo5x8zzTt68WJfjJhrHimS0REFBI2ukRERCGpWN3LhUgd53cn1R+X2LJ3LK7pfnBgwfm09z7OtPw0PbEVIazq18iZjvYUqqe+O9qZbo+JBeaj8uP5nm840ytz/atk6z9ePezqUIgaPu+/BOPA48/34ok93ScTXnNDlhe3/Re7l4mIiCo0NrpEREQhYaNLREQUkkozpptUEQ+gSolyLMMx3OTbWa/gp4EBwNRs/ylEnR9a5qTx5eXl07J/H+LFvTPc24B+yfbHcVN5i1DFluffbFT/EX+9r33DfRLZ7HP9p5T1f/tCJ02nzkxS5Vw80yUiIgoJG10iIqKQsHs5ESJeTh/tJfaUfI2OWB41bczmfb04d83aMKpDSTbgvG+8OPJF9ZdMuciLW8F92Uhq/Xr+RKP6Xpg7e35iK0ihS/l+mhf3fW2okzbrH3738pb73K7nWmf5t34m8+mBPNMlIiIKCRtdIiKikLDRJSIiCgnHdBMgr2r0Mdw1udkh1qRykgz/rVGn7PVb1HzrdmV6sWZzvVR0ebn+OcXqwYc4aSde+qMXj17Y1IvLwkvOKXHavbDUmX7jrCZe/EP3kU7acT3+4cUp45N3eyfPdImIiELCRpeIiCgk7F5OgDePe86Znr3L724+79UbvbglJoRWp0ol138azQuzD3WSrj1ksRePW9rOi5shnKfPUOmZffgrXpx3uHs7Udcf/K7Edndu8+JYX6JO5UPOUvfJc++f1seLB379npO2duhOL240Pnl14pkuERFRSNjoEhERhYTdywlw96KTneltzzTz4paj2KWcbJrjv64g6+ZtTlrnBwZ6sUyvCapYvrjV7y6c9e+mTtrPEzt5cacnVjhpbf+e68W5O3eCKofgE8fOWXiMk/bJvi958SUHXeUn/DIjoXXgmS4REVFI2OgSERGFhI0uERFRSDimmwhHupel18CyKBkp2XL/XORMtzyrlCpCoaj6ySQvXvOJm9YOv3hxDohc209zbyObOGEvL97QsYYX1/0FCcUzXSIiopCw0SUiIgoJu5eJiKjSyV27zpl+oUMbL66Ln5M2X57pEhERhYSNLhERUUjY6BIREYWEjS4REVFI2OgSERGFhI0uERFRSERVi85FREREceOZLhERUUjY6BIREYWEjS4REVFIyn2jKyKvisguEVkcY/4MEdkqIrtF5N4oebJERG2+yxNa4QQTkQ62nrkicmlp1ydeInKnXTdbRaRG0d8ARGSB/Q28WUgeFZFtInJf4mqbHLEsT1lWgm2yyN+wiPQVkTyb77iEVjhEsex/yrrKvo3Gu88tE42uiIwTkZ12QbaKyNxiFvGwqmZFlHmUiPxqV+IyETkbAFQ1W1UzAbwVQ7l1VPUFW96AQP22ish2+yPpadOvE5GFIrJZRFaIyGMiUuCzrUWkioiMFJHFtoy+Eenni8hKm94v8HlbEZkgIqn5n6nqPLs8P8b0lwqBiHQWkW9FZJOI/CkipxWziPdUNVNVt9ny6ojIayKy2v67M5hZVdsCuD+Gcnuo6q2Beh5hfyOb7bq7PJAmInKriCyx6e+KSK3CCheRa0Rkkf3NzRaRDvbzHiIyU0TWisj1gfzpIjJRRFqUcHmSRkTqichHdln+EpHzi1mEs03axmaE/Vv+Hfw7FOM3vML+LsbaMpuKyBi7vamIZAUzFzZPm36kiMyx2/J3ItIq2ozFHIh/Z/POEZGjIspZZOdxbuDzOvb3VTOwrMXZ/ySViJxrf6fbbKN4WDG+7myjtrz9ROQHu39cJSLX5KfFsY32F5E/bJkTRKRLIC3D7mdXiMgGEXlGRNILWd78Rj1/H/5SIC20fW6ZaHStwXYlZqpqx3gKsivmbQC3AqgNoAeAqfGUqapvBeqXCeAqAAsB/GqzjAGwn6rWAtDNznNIIUWOB3ABgL8j6p4G4EEA+wEYDODJQPJwANepam48y5JMtv4fA/gUQD0AlwN4M78BKqHHAFQHkAWgF4CBInJxnPVMB/ARgOdhfiPnAHhURHrYLBcCGAigN4C9AFSDuy4iy7sUwCUATgSQCeAkAGtt8gMAboD5TdwqIk3s59cDGKWqS+NZliR5GsAuAI0BDADwrIh0jaO8OwG0B9AKQD8AN0r8Z6x5AMYCOKO48xSRBgA+BHAbzO90CoD3CpnXOwCmAagPs18ZKSINbdrjAPoDOBbAM4Ed9AMAHlTVLSVZuGQSkaMBPATgYgA1ARwOsz8raXkNYNbF8zB/o3YAvoyzju1hDk6uAFAHwCcAxoh/MnMzgP1h9rcdYPaZw4ootkdgP36pnU+o+9yy1Ogm0jAAz6vq56qao6rrVHVBgucxCMDrau+5UtUFqrrRpgnMDqFdQV9U1V2q+riqjgcQuTLrA1iuqisBfA2gDQCIyJn284kJXo5E6wTTSD2mqrmq+i2An2AasJLqD3PmtF1VFwN4GcA/4qxnPQC1ALyhxmQAswHkH0n3B/Cyqi5V1a0wO6hzRKR6ZEEikgLgDpiNc5Ytb4GqrrdZWgP4VlWXA5gPoKU9qzoD5oCiTBHTZXgGgNtUdav9nY5BfOtwEIB7VHWDqs4G8CKAi+Kpp6quUtVnAEwuwTxPBzBTVT9Q1Z0wDXQPEekUWYg9YNwPwB2qukNVRwH4HX5jX0NV/1DV32AOVOqLSC8ArVX1/XiWMYnuAnC3qv6iqnmqutz+PkvqegBf2JOTbFXdYv/m8TgWwI+qOl5Vc2C2wWYA+tj0/gCGq+p6VV0D00CWZL8Q6j63LDW6D9jut58k0N0qIi1FZKOItCxGWQfZ7/5uuwzeFJF6iaqo3WEeDuD1iM/PF5HNMGc4PWCO+oprDcxG2xzA0QBm2u6pYQD+HVfFS4/AHI2aCbM+Dy1BGQWWVxKqugrm7OViEUkVkYNhzojGFzLPDJgzp0jN7b9uIrLUdjXeZRtjAPgDwDF2nWYBWADgCQBDVXV3PMuRJB0A5KjqvMBnvwHoChR/mxSRugCa2jL2KC8ZYphn12Ca7SZdEKVOXQEsjDhjDZa1WswQQg+Yg+0NMOu3sJ6uUmPPxPcH0FDM8M8yEXlKRKoF8hR3Gz0IwHrbFbtaRD4p5j47anUj4shtPzK9uYjULqS8H+wwwIfiD0eEus8tK43uTTBHF80AvADgExFpCwCqukRV66jqkmKU1xzmqPwMmJ1koV2DJXAhzBHYouCHqvq27V7uAOA5AKuKW7Cq5gG4EsBImC7Jy2COSp8EsLcdV/pCROJqdJJoLoDVAIaKGbM8BubI1DtDtOtzfLQCCjAWwM0iUlNE2sEcze5xxlkC7wC4HUA2zPjMrYGu3rEALhUzllcb5jeKKPNtbv8/BkB3mK7M82C6mwGzHq+EOVu8DqbLeguARSLysYh8LyJnJWB5EiUTwOaIzzbBdEOWZJvMDJSxR3lJUtQ8MyPSCqtTUXmvgGlkX4DZ71wJc8ZU1W6r34lIH5QdjQGkAzgTwGEA9gGwLwJdsyXYRpvD9CxcA6AlgEUw21c8vgbQR8xFdFUA3AKgCvxtcCyAa0SkoR2yyT/IibZv6ANz0NsJwAoAn4pIWtj73DLR6KrqRNsdka2qr8F0R54QR5E7ALxiB7y3wgzgRy1PzEUu+YPrsVxMcCGA16Ilqup8ADMBPFPMeud//xtVPUhV+wBQmKPSV2HOrC8CcA+Al6IWUIrsmdupMGObfwP4F4D3ASyLo9ghMOt0Psx48TuFlScinwfW54AoeToBeBdmXVaBOWu5UUROtFlG2PmMg1mX39nPC5rvDvv/w6q60XaBPw/7m1PVv1T1BFXdz9b/HpiN+78w44gnw4wnJ6w3Jk5bYbreg2rBHCiUtLz8MmIqT9yLFktyxlTUPIuzjIXmVdXpqtpXVQ8EMAvmoPB+mG30Lphx0zdERFA25P9en1TVlaq6FsCjiH+f+5GqTrbd9XcBOCTaWWcs26iqzoFpyJ8CsBJAA5i/b/42eB/MOPt0ABMAjAawG1FOdlT1Bzu0txHm4KA1gM42LbR9bplodAugcLsNimuGLSNYXvSZqXYNDK4XekWaiORfWDOyiDqkAWgbS2ULmZfA/OCGwPzgUlX1L5gxrL3jKTuZVHWGqvZR1fqqeixML8akOMpbr6oDVLWJqnaF+d1GLU9Vjw+sz2hXiXYDME9Vv7BjWnMBfAbgeFtGnqreoapZqtocpuFdbv9FmgszlhfLb+52AC/a7u3uAKao6iaYHUmB1wCUgnkA0uyFLPl6wPwNik1VN8DsNHsEPi60vMD6yyxmL1es85wZTLPj2G2j1GkmgDYSuAq5kPo/BmCYqu6Av34Xw5xZNiwgf+js32YZirGPjEFx97mxbKNQ1ZGq2k1V68NcN5EFO4Zvx9cHq2ozVW0DYB2AqfbMNRZ7tDNh7HNLvdEVc1n9sSJSVUTS7FHP4TBdByX1CsxYXRt74cvNMFfTJsIgmCtOnSNiEblURBrZuAvMWMA30QoRc7l7VTtZxS5/5IHGpQB+VdXpMD+oarbsfojjSsNkE5G97fJUF5EbYMbWXo2jvLYiUt+OvR4Pc0V0vPc4TgPQXsxtQ2KHM06C2Xnk3zLT1qZ1gTkTuLugDVpVt8Ocsd5ou8Cb2zo6vzlbTl8Az9qPFgE4QkQawwyDFLtxSQY7vvkhgLtFpIY90DwFwBtxFPs6gGEiUtf2MlyGOH4T+ew2lGEng9tUUfP8CGYM/gz7ndsBzLBnVw47tj0dwB32d30azA54VERdjgZQVVXz13v++u1q67gu3uVNoFcA/J+INBIz/n0d4ttHvgLgNBHZR8ydAbcBGG8PKEtMRHra7b4hTPf9mPx1JCLNRGQvu40eZOd5R5Ryutq6pYpIJoBHYA6gIy/2Sv4+V1VL9R/M0d9kmK6ajQB+AXB0IL0lTPdOyyjffxXAvQV8fhfMAPkamJ1F3Vi+Z9OyYI6C0iI+r2rreGQB33kFpltjG4DFAP4DswHmp88EMCAwvdjOI/gvK5DeAOYCnFqBzwbAdNkuBtAvYv7jAFxa2uvT1uU/MBeTbAXwOYB2EelbARwW5bt3Angz4rOzYcZgtsPs/I6N5XsR6VpAPc62f+MtMEf+DwFIsWkdYM5gtwP4C8D1Ed99DsBzgelaMN3VWwAshdmJS8R3vgNwYGC6B0x32doCyi90eUJYh/Vguuu2wRwMnB9IK/Y2CdPojIAZK14VubxF/YZhDlaWRVmvzr9Y5wngKABzYLpGx0Vsf5HrN8vm2WF/F0cVsHzTAbQKfHYkzLa6EsC5Rf2NQl6/6TDDXxth9inD4e6virWN2s+vhGnINsDc3tOiOL9pFLyNjrfb1HqYIZsagbTD7d93u10nAyK++zmAW2x8hM2zDeaak9EA2kfkD2WfWyorPME/nhftD2RBjPkz7A9tG8wtAAXlaQVgp813WWkvYxHL097WczuAi0q7PglYnmF23WwMbmBFfGeu/Q2MKCTPTpiLX+4p7WVMxPKU5X8l2CaL/A3bHewOm2+Pg67y8i+W/U9Z/1fZt9F497l8tR8REVFISn1Ml4iIqLJgo0tERBQSNrpEREQhKfAtOMlydMpZHEAuJV/lfZDwG/O5PktPMtYnwHVamriNVizR1ifPdImIiELCRpeIiCgkbHSJiIhCwkaXiIgoJGx0iYiIQsJGl4iIKCRsdImIiELCRpeIiCgkoT4cg4iIKp+U6tW9uOcE51XkuKPhdC8+ZtbpXlzl6L+SX7FSwDNdIiKikLDRJSIiCgkbXSIiopBwTDcJ0po09uJd7feK6Tvp85Y703P/3caL68zyn5tdb/ZOJ1/Kj9NKUkWicmNn/17OdLXPf/Vi3b+LFy86uYaT77AjfvfiH7/tHrX8pj/nenHVTyaVuJ7kCo7jznuhoxePbviCky8vEC/9rakXtwXHdImIiCgObHSJiIhCwu7lEtp0wUFevO4Et8v35n3HevGFtf4XU3kvb2rpTJ9e8yMvrntW1ajfO6lZz5jKJyrrUhvU9+Lc96p58bvtH3XyrcpN9+LaKeO8uGVadUQ16IeoSasv2O7FK4ZXcdL+ef81Xlz/xZ+jl097WHhrDy+e1W+4Fw9YeLyTb919rb247dhfkl+xUsYzXSIiopCw0SUiIgoJu5cjpPTo7MVz/s+/GvLHYx538jVMnex/JwHHLpfUXhLxSfQuZaKKaN4T/hDL3E4vB1LcbuNGqX78zMYOXvzrFneIZtm2OlHnlSr+NbOfdfykwLIB4L1h//HiK2YPdtJSxk8HRberUU6Bn8/4sb0z3Xps5eq255kuERFRSNjoEhERhYSNLhERUUg4phthW+uaXjzv+GcDKdX2zByn5zb6T516668DSlRGbfyZqOpUeCn7+E8v2tnEfXrR4lP9p36d2Wuyk7Zb/YG+797wn47U9PtNTj6dNjMh9aws9OAezvR7hzwfmPJ3TWN3uGO6Dw4d5MU1Z671E9asd/KlbFgafd4p/jrt8MhVXjzr7CedfG3TM714x7DNTlrti/wnz+X8vSrqvCqr9MxdXrwlz49bfpVdGtUpM3imS0REFBI2ukRERCGpsN3Lac2bOdOzb2ruxY0n+F2Jtd5xn4CSkq1ePG+33yWyNMe9/aBF2kYvvuiPQU7ahtn+k3UaT/bLqzPB7e7SrVu9uOIAnrsAACAASURBVPZGdhMngvbex5leeLUfv33wi17cs0rEvSGxGuo/EH/HDbucpBc2+t3Xz/zWx0lrf8lsL87b6T7BrLLaXdt9+tM+VfzdUR787WboK/9w8rX4aIIX56KE8vxvtrvO3wd0ruLeFjTjlCe8+PvuI5203kf53dK132T3cmq71s70zMNHePE1K4708333KyoznukSERGFhI0uERFRSNjoEhERhaRCjemm1qntxb0+W+SkjW4wxot7T3HHbYIyPvdvFxl64kVenDtzrjuvzv6jzOrNXeCk1cubV2DZBT8UjUoi71B/7HaxP7SGz3o/7eRrmxa81csfx/1qh3sL2C2zTvXijUvc8fs/TvVvI7ltlf92qYebTHHy9ajmv3T70V7vOWn/vu4iL27+wAQQkFtVoqbtPeEiL255X3h/r/ZXT3SmPz3Kf6n6WZnrnLSNJ2/z4tpvJrde5cHcO6M/djNM2cf7t19uaRG9iWs41b0FTKeGc8sfz3SJiIhCwkaXiIgoJOW6ezmlqvsmnuyRfvfyLQ2+ddI6fuj3QXb6yO9GKOyWg8guZSdt9vwYa0mJsPBt91agt6Le/uN2G5+36GgvnjzHv6Wh0zWznXwNt/nrumHEvK/oeZQXrx7Syouve9a97WhY43Fe/OOOpk7a9MF+F/Wpb57ixTlLl6Gy6vjv6N15qVNrRk0L062T/WGHs/q97KRd3fUHL/4UdUOrU1n12IHvRU376e39vLgJ4h8uWPDWvs70Ewe+48Xdq4z34sapGVHL+HO3O+B3ysjrvLjtDb9EZk8YnukSERGFhI0uERFRSMpd93JqXb8bZ849HZy0uZ2f8eKpEc/U7nT3Qi/O3exetUZlQ0oN9yUE8+/u7sWz+7hXJacErkSeHHiK2ICPr3bydbzL70busNG/2jgPsetec7kXf5Xmd1FP+U9PJ1/9R/0rX0+tsRGu6FfqViYpe3fy4r51vnLS5u32n9TVYMbu0OpUmLrfB4aw+pVePcqq1Fq1vLhGirvT/XKHvz03eSy2LmVJ959Stqvf3k7arc++4sWHV53qpKWLvz+YlO13KV845ywn3/Wtv/Tik2tsd9KeOdUfPnh8xGlenDur4LtRSopnukRERCFho0tERBQSNrpEREQhKXdjuisu6OzFc09zXzg9Zps/3vvySUc7ablr3KdGUdmz8eTuzvS3Z/3Xi1Pgvsj8mx3+uM2DV/lveWr3pXupf6xvoZE0f1NI6djWSXtpdD0v/s/rr3lx9yqrI0rx65gq7vFs94nne3Gz1ZX3tzh/kP/UonMz1zhph84Y6MW1/jcZVPYturabFx9a9Rsnrct3F3pxO0yLWkbw7URzr27sxbPOfrKg7ACAb3ZkOtNXfXGRF3d6Yq0XZ8xzt7Wn4V8H9OQ3LZy0Tzt96MUPtPRvP60yK2o1SoRnukRERCFho0tERBSScte9vOXAHVHTnljkvyi52rzK24VXXmnEe+V3avTbbLbk+U+e+vtA/zaDHaf3cvK1a7+ywO9v2uk+zeysVv6Lta+u84aTNmWXX37vjODNRm6Xd9BPO92bkprd6y+LZmdHZq80rjv+My8O3iIEAFWerh+Y4vZbHsje0W+/TF9QLWpaUPBFCXP6+bcGRt7WN2Dh8V68+cZmTlr7n/3b9WIdUvpzYRP3g04F50s0nukSERGFhI0uERFRSMpd9/I7vV8ITLnHDCO7+C+1PPjRfzlprcfs8uLUcb+Cyp66H7sPwL/8wgFe/GYn94WlJ9fwn0J1xpX+k8hyNfqzprLVf8B5hhT203fT3C5lX05ER1bfGed6cb2r3TRdGM67OsuT59cd7kxX/XRSKdWESqpTo1XF/o707OpMf3Tos4GpdC/qOu5yJ1/7S/yny8nO34o936Lcvtp/D2/Vcb97cXGeXhcLnukSERGFhI0uERFRSNjoEhERhaTcjen2yvD7/HerO25WN8W/DWTOOe5baXaf7eft9s0VXlx7snvryNbm/lhhLf/FRGgwY1vUOq3d2307TuNx/pOKcnnrUszytmxxpjOO8acvb3y6kzb7ziwvPqanP/4yb1MjJ99fyxt4cWoV/zdwcscZTr6Hm0xBcXX5zh1z6vgv/21EOasin1ZVOaXWqe1M10xZVko1oWRoXt1/m1ZK5DmcKAoyb4j7YvnO6f4+vefkC7y47QD3KVaJHltNz9zlTG/L8euVt3NnZPaE4ZkuERFRSNjoEhERhaTcdS+3/uQyL5530nMxfy/4kuO5R73oJxyVkGo5Jt3sP33o2lmB20hOSuzLkCuT3Iju2g5X+tOLA59XwV9OvvYR0/m+/KiLM11Y9/LiHP9l16c+eaNf9uPuLS65OTkg17JL3NtDBtT8zot/3ZYVcm2KL/uETVHTtudViZpWWeSpf96WF9kBHOWJck0bb3Smg9/r0tC/BWlDAuoXKfhyhZmHj3DSDp9xthfXSuIT0XimS0REFBI2ukRERCFho0tERBSScjem2/Fq/zLyYz9wb9m48KlPvLh6ivsml5Oq+y/MDo7vJkOvDP9S+fH7vuXFXf8zxMnXdujPSa0HuRbdf7AX/3rAYxGp0cfnznzYH8fd6+kJXlzwDRFUnuUc0dOZfnffpwJT7q0uHz3kv9WsNn5JZrUqlDqXuLfjTPzRv2XoqZb+Pvzgh25w8nUY7l+fkbN8RYnm3fk9v4xVue4b66o+US8wxTFdIiKico+NLhERUUjKXfeyBm7LSP96qpP2Tqe9on5v+Jn+rTu56f6l7Ifc4N728WCTyfFW0RF8SkvzHgW/UJ2SZ8XQQ7z4iwEPe3E1if4C+ic2tHOmm7wy3YsT/VQcKn3BLuX117hPnuuU7ncpX7W8t5NW5z3/bWWVZagheMsNABxe+9tilxHZNfzQUad6cY9R/mMA/7hguJPvqj79vHjlifWctNx1671440B/GOnQayc6+W5v/JMX93zX7b5uOzacIQKe6RIREYWEjS4REVFIyl33cknVGDmxwM8/6XGwM/3gQL97ebv6D8Tu+cOVTr5WL/lXQK8dst1Jm3KA+8J1Cs/uY/Z3pkcP9ruUW6ZF71JeEnjq1JibjnTSMrYndsihMqm12H0pSfDpXqVJ0vxd38br/BdrTNnvXSffVzuqefG829yna1XZXfyXZJR3uX8ucqbf/buXF5/WdqyT1urQJV6cWquWX8bmzU6+nIWLvXjqvv554OED3bs96s3wn2QlDXY7aYueauHFMw/3rziPvEI52KXc9obSueKcZ7pEREQhYaNLREQUEja6REREIak0Y7rRtPzCfXIVBvphdfGfUjS7z8tutlZHe/H/sr6IKLXgY5klf7uXubd33o9DibD4JPdpY1lRxnFX5rpjixde+y8vrv5ZweP/VHw1Rrl/y7H3dPbitlXXOGnzm3fz4pxly+Oed96h+3jxoqvctDM6+7eB3d/IHccNuv+GQV5c7YtJUfNVVjsv9cdqHx3VyUn7tNPHXnzNN/7tVpOec6+jyVxR8Nu51hzg3qB3wBD/dqJH9hrvpAVvzXxhU5YXv/rfk5x8bUeU/lMAeaZLREQUEja6REREIan03cvpU+Y70wf9ep4X/7LfO1G/90bWV4Ep99glW/3L2U8KvMS+0xD3IdruzRRUUqn1/W77aac/HpGagYL0HT/YmW77EbuUw3ZVHff2k1Wf+l2VU9a3jLv8B1u/4MX7VIm+q5u6y98SB066xElr++0cL+b2uqfcef4+7YdT3Fuq6n7mP93rsb1+9BPu/hHRBLuJ84rx/Ldu4y/24nbXr/XiestLvzs5Es90iYiIQsJGl4iIKCRsdImIiEJS6cd087Zscaab/F9dL+4/4mQvviXrMyffwRn+CM+orQ2ctFv/d44Xt7vOf9QYx4QSJ7Wuv56uneiPEWVKwWO4APDQOv92lfaXuWP5fHtQOIK3cKy+5gcn7a6Gv/kTwbjE/N1bTsTW95v/hFdc8J7/uMHWN7tjgNxmYxd8nCMAjO7r3wI2/GL/TULbWruPcPziOP86jGO/uNZPKOTVTR1f2ulMZ02e4dcjlsqWIp7pEhERhYSNLhERUUgqffdypJzF/psxcIQfDhniPtJmywH+2ys6DVvrpLX7q3TeXlGZrD3Zf/rNMdW/8+LcQrqk/ndXXy+usY23CJWGeoEnAk3+oYOT9uhov8vw+rpu939JdPr+H15c5Xf3yWTNH5jgxa1R9m4rqQhyV6324mYPro6a7//gP62qA2J7o1chm3mZxzNdIiKikLDRJSIiCgm7l2PUePgEdzoQl/Wr5SqiM2742otzNfq1x+0+ucKLO4xil3JZEvlC9K+71fRj7Bd3+W0wvehMRCHjmS4REVFI2OgSERGFhI0uERFRSDimS+VSj2r+rV2p4h87/rLTfYZQl4f9WxX+v707j5KquPcA/v2xDatsCogswyKyqICIa0AEDeoLRKLEXXEjJCp5aohGjCCap4nnKYriCiruEkXAF1EjEDcQQWQTMLIpLkBQ9p35vT/qTt1bTXdPb1M9w3w/58w5vztVXV23q++tvlV34dw7EeUbj3SJiIg8YadLRETkCYeXqVz67xfCh40vu3asja8af4OTr/lK91IvIqJ84pEuERGRJ+x0iYiIPGGnS0RE5AnndKlcajkinKvtO6KLjZuDc7hEVHbxSJeIiMgTdrpERESeiGp5fhwwERFR+cEjXSIiIk/Y6RIREXnCTpeIiMgTdrpERESelPtOV0RGisheEdkmIrVSfM0KEdkjIs8nyaMisl1E/pK72uaeiBQE675XRO7Od30yISLPBO2xOsX87YJ13i8i1yTI00tEioJ8Z+W0wh6Vx/at6NtkSURkuojsEpEP812XTFT07VVErg7qqSLSNt3Xl4lOV0Q6BF/EzSLylYgMSLOIV1S1tqpuD8qrJyLPisj64G9kNLOqtgHwPymU21lVh0fq2U9EFgcf+Mci0jGSJiJyt4h8G6zHTBHplGSdV4vIzqCsbSLyTiStj4isEpEfROTCyP/richnIlInsi67VbU2gBdSWJ9SIyINRGRSsFNcIyIXp1nE31S1MFJegYiMF5EtwedwU3Gaqn4ZrPMHJZT5XfC9mBaUebiITBGR74INpjCaOdl7Bul9RGSZiOwQkRki0jLRG4tIYZBnR/CaM2LKKdPtKyLXi8hcEdktIs9kUETsNnl68HlsjrezzmKbfEJElgc77EFx1uPG4HPeErRtQSQtYRvFKSfhd0NEmovIbBH5UUT+N+Z1b4nI8THr2hvAkBTWtdQE+6ddkf3P8jSLiN1eizvibZG/ykBW26uIyHAR+Tr43F8WkUPirEsDEdkgSX7EiMhjMXXbLSJbI+mjReQnEZklIs0i/79YRB6KlqWq44L1yUjeO10RqQJgMoA3ATQAMBjA8yLSLotiHwBQE0AhgBMAXCYiV2ZZzyNhdnxDANQDMBXAlKD+ADAQwFUAesCsxywAz5VQbL/gS1ZbVX8e+f9oAP0A9AUwtvjLC+AeAPeq6tbYgsqARwDsAdAYwCUAHk32oyMFIwEcCaAlgNMB/FGy/wVcBGAagPPSfU8RORTA6wD+DNO+cwG8kuS9XgIwH0BDAMMB/F1EDgvSykP7fgfgbgDjc1Te9qCsYTkqr9gCAL8D8Flsgoj0BXArgD4wbdoawJ2RLMnaKNZIJP4+/gnAswBaATi3uJMVkQsArFLVuVmsX2m6PrL/OSoH5f0tUl5tVd2fZXmXA7gMwKkAmgKoAWBMnHx/BbA0WUGqOiRaN5i2nwgAInICgG4AmgD4EOY7AxGpC/N9vT3L9XDkvdMF0B7mA31AVfer6nQAH8F82JnqB/MF2KGqqwGMg+kQs9EXwAeq+qGq7oNp6CMAnBaktwLwoaquDL5szwPoGL+oEtVS1cWqugCmI2sYfDFaqeqr2a1G7okZQjwPwJ9VdZuqfghgCrJrwysA3KWqP6nqUgBPAhiUTT1VdZ2qjgXwaQbv+SsAS1R1oqrugtkJdxaR9rGFBD8YjwMwQlV3quprABYh7OzLfPuq6uuq+gaAjTkqb46qPgdgZS7Ki5T7iKq+B2BXnOQrAIxT1SWq+hOAuxC0ZwptFK+sRN+NVgCmq+pmmO9W6+CI7FYAt+VgNSuqfjDt942qboPZ514gIjWLM4jIKQCOBvB0qoVG9lfPBv8q3nfvBvAezI8zAPgLgPtUdUvWaxJRFjrdeATmgzQLIptE5GcZlBG3vCzrFY2j5b4MoI2Y+YuqMBvptBLKeyEYFnlHRDpH/r9eRDoH/ysC8BOABwEMzcE6lIZ2APap6peR/y0A0AkARKRF0IYtUilMROoDODwo44DySkMK79kpmhYMm65IUKdOAFbGHLFGyypv7XuADLdJ35w2C+LGItIQJbeRlcJ3YzGAM0WkHswR0xKYDn60qm7K0bqUhntE5D8i8pGI9Cr+Z7rba8TvgiH2eSKS6MdLumL3uQUwIw4IRogeBnA9gHTu8nQegA0A3g+WlwDoISI1YEZFlgSjFUep6ovZVf9AZaHTXQ5gPYBhIlJVRH4Oc/Rof82oar3g6ClV0wDcKiJ1xEx0XxUtL0P/BHCamAn/ajC/YKtFyv0eZmhiOYCdMMPNNyYp7xKY4e+WAGYAeDvYaAEzhP0ggCdgjhZ/G7x/dRF5O5iHOu3AIvOmNoDYX4ObAdQBAFX9OmjDr9Mor7iMA8orJSW9Z+2YtGR1KilveWvfA2SwTeZDbDsUx3XipBWnJ2rP6Otj894DM630LwBjYfYLxwKYKiIvisj7InJ9pitRSm6BOaI7AuZ7OFVE2gAZba8A8BBMZ9gIZgrmGRE5Ncs6TgNwjZi597pBnYFwnzsUwCeqOi/Ncq8AMEGD2zGq6mIArwGYDaAFgL/BrM9QERkatN8Lkf1zVvLe6arqXgDnAvgvAD8AuBnAqwDWZlHsUJiO798w88UvJSsvONmheIL9kgT1XAbTWA/DdLCHAvgiUu4dALoDaA6gOszc0fToUEhMeR8Fw1o7VPUeAJtgNlyo6ueq2ktVTwze4yqYk0yeCsq9EsBzIiLxys6DbQBiT3A4BECmc5PbImWkVF7MSRLp/kJP5T3TWcekecth+3qXyjaZgth2KI63xkkrTk/UntHXO3lV9UdVvUBVO8P8mBoD4AaY4eXFAM4AMEREOmS4Hjmnqp+o6tbgRL1nYab0zsmivM9UdaOq7lPVf8Cc//KrRPlT3F7Hw+y7Z8Icjc4I/r9WRJrC7OeHx39pwvdtAaAXgAkx9X9AVTur6gUAfg1zFFwJ5hyjPjBzxrem816J5L3TBQBVXaiqp6lqQ1XtC/MLbE4W5f2oqpeoahNV7QSzngnLU9WzI5PsCc8SVdW/q+rRqtoQwAiYI9Xi+cEuMGdsrg2+eM8AqI/U53UV7lBKsQcA3K6qOwEcA2BuME9dFUCikz58+xJAleBks2KdYTaUtAXzb98HZaRUXswJHOn8Qk/1PZdE04J5oTYJ6rQEZl4vetSUqP7loX29S3WbLIHTZkG8TlU3Io02SvP7OBjA7ODoqbg998DMFx+T4Xr4kGj/UyrlpbK9qmqRqo5Q1UJVbQbzeX8b/J0AM+T/hYj8APNj5wQxZ5ZXjlde4DIAH6lq3HMLRKQxTBuOgpk6XBgcGH4KM3qRtTLR6YrIsSJSXURqisgfYD7MZ7Ior42INBSRyiJyNsyHmPU1jiLSLSjzMJghmSnBETBgGmWgiDQWkUoichnMjvOrOOW0EJFTRaRasN7DYI6cP4rJdyaA6qr6ZvCvVQB6izkruAA5OsklW8H85usARolIrWBY6Zco+eztZCYAuF1E6gcnK12LLL4TxUSkOsxnBwAFwXIq7zkJwNEicl7wmjtgNshliBHMbX8OYETQvgNgNtjXYupSZttXRKoE61kZQOVgPTJ+/nawTVSH2SYkKK9aDupZLShXAFQNyi3er00AcLWIdAyGBm9H0J6ptlFEid9HEWkE4DqYk+wA056ni0htAMcjxyeRZUrMpWl9i9s0GEnoiZLPQUlW5vkiUjto558DuBTmZMps6tkg2JeLmMsz7wcwSlWLALwFc9DTJfi7A+ZM9C6a/Kzpy5F8P3I/gJGqugOm/boH7dcLuWo/Vc37H4D7YE4m2QbzYbaNSd8GoEeC144E8HzM/34Nc8nDDpgNq28qr4tJ1zj1+BBmSOlHAI/DnIVanFYd5rKZ72HmNz8DcFYk/TEAjwVxJwALYS6j2AhzxtzxMe9VENS9ZeR/fQCsDt7jwpj8zwC4O49t2ADAG8E6fQ3g4khai6ANWyR47QF1D9Z/fPBZrgNwU5zXzQRwTYIyewFYm6Bdnb9U3xNmmHAZzNTFTACF8do3WC4M8uyEmec/ozy1b7B9xH5WIyPp6W6TveKUN7Ok18Vpu9htcmaccntF0m8K2nILzBmuBam0Ecw5F0vS/D5OADAwstwcwCcw+7b7Y/IOgjljNh/b6mEwBwlbYaa1ZgM4M5Keyfb6Acw89xaYk8wujPO6mUhje4U5QXM5zH58TbzPPNHnGW8dAJwMs3+qk6CM3gD+L+Z/o4P2mw2gWUnfx5Q+/3w0eo6/QLcHH+QmRDrBEl6zPGiQ8Uny7Aq+RHflex1LWJeCYN23w1z+kPc6ZbAOTwbtsSLF/EcG67wDwKAEeXoGO9NNiPOjq7z8lcf2rejbZArr+m7Q4b2X77pkWP8Kvb3CnHOxKfg+tk739XyeLhERkSdlYk6XiIioImCnS0RE5EnGZyNm4sxKAzmWnSfvFk3M+TWfbM/8KY32BNim+cRt9OCSqD15pEtEROQJO10iIiJP2OkSERF5wk6XiIjIE3a6REREnrDTJSIi8oSdLhERkSfsdImIiDxhp0tEROQJO10iIiJP2OkSERF5wk6XiIjIE3a6REREnnh9yhARUS599cBJNl5xwWNO2uVretp43clbvNWJ0rOvdzcbrxoQdkk39/mHk29w3dU2rgT3AT5FCB+mNGJ9VxtPXX20k6/pPZXDhTmLMqpvtnikS0RE5Ak7XSIiIk84vEwHtSpNGtt486mFNv72TPfZ3qv6P2HjvbrfSTv18wttvOGb+jbueO8PTr59q7/Oqq6UvlNP+iJh2oSW79u4x4DfOGk1J31SanWqqL695RRnefuRe2x8Ubc5CV93Z6Nw2ytCkY0rxRwTRtM6zBzspDWaUmDjOq/MtnFTJP5+5AuPdImIiDxhp0tEROQJh5ep3JOCcGhp5Z3HOWkPn/+UjU+rsSNhGXs1/P0ZHcYCgA+6vBgudImEDa9y8rUYmFJ1KYeiQ8jJfNfTPdu17aTSqE3FtmDow85y9Izidft32njsRncYut1b4dB/rX9Xs3H1/7hTQA3HzbJxG8zPrrJ5xCNdIiIiT9jpEhERecJOl4iIyBPO6cbY3yucE6xyxzobTz1qipOvqoR3Nkl2iUnD4VVtLKu/dfJt7NfRxg3eWOykFW3dmk61K7Svh4V3tFl02YMZlXHlmj42Htfy3ZRe8/kp453l/uie0XtT6Wt74+ySM1FWei4631mefswrNo7O487r6h7rtcPc0q1YGcMjXSIiIk/Y6RIREXlSIYeXo5eYbO3fxUkbcU84ZBi9xMS9iATYGzmbPdklJsf9eZCNOzdxf+NMLgxPse9e7wYnrfGYj+NXngAAenJnG4+/akzarz/26aHOcqu7PrNx+weuc9KW/fKRtMsnqmjqXbvHWX7zvYY2PrfePBt/3uFiJ9/+pf8u3YqVMTzSJSIi8oSdLhERkSfsdImIiDypkHO6u3sdY+Ppox9OmG/Gzto2vuNu95Z/VXdobHZrS8vwt0y1yJ0H//gH9xKTzUX7bFz7e/eyI3JF53ABQO/+0cbdwin6A+beJ21rZOPxg/rbuPAT96knWhR+/kfduMBJO/uN39r4rsfCJ6IcX+C22RmLw8u8/nl0ndhVoFLQ5pUhNo59iH1U9GH3AC8hKg37vlnrLN866RIbf3FpuJ/d08TdNiovLd16lTU80iUiIvKEnS4REZEnFWZ4OTo8ec+jjyfMd9GKc2y8ZURzG9efMSte9rjqtm1l4y4TV9i4QzX3N077yTfauN3f+VDtZNZ3r+Usf9o+HKqP3h1sc5F72cKIV8O7gxXOSq0NdfduZ7nqO+Edcy59OxzOXNLPnZoY1iBs6ydfusJJa3WRO2RNuZFsSJnyLPJgp0qRhY2dqjvZGkg3pKJgbnhp0f4tW7KrWx7xSJeIiMgTdrpERESeVJjh5Z+Ghw9Rjp7tes6yXzn5Kv/hkDCe/xkysalbYxuPaPRqwnzN38mo+Aqp0hkbneXoXcCidwe7cmV/J1/hn1OfFkhFu9+GZz2P+VknJ+2mBstsfEnHT520j1ENRAezKs2bOcv3nvuCjaMPtJ/9J/ehJJUix37R7bpSzDFhr0UDbbx7orvtRR9wX9bxSJeIiMgTdrpERESesNMlIiLy5KCd01318rHO8pKuT9t47b5wfrfS8PpOPp2/MO33ij61CADa/vcXYfmR3zXRB6UDQI033LsikavKEU1tfPNR/0zpNSsnHuksN8aGnNYpavzkM5zlm65cliAn0cEpOo97ztvuZXH9a/1k4xHru9p46uqjnXw6u17csvtf+KGzfFPrcB9w7qhNTlrRqHDO+KzLBts4epkRUDYuNeKRLhERkSfsdImIiDw5aIeXL+/oDt1GT0Vfsy+8LAiz0x9OBtwh5eWj3ZvxT24RPvQ8egP+Nfcd5eSrCd6FKpmfftbCxufXnpww3+Bvetn4iMgdwABgH/Lj6Bruzd/ntO5t430rV3uuDVHp2NYlnAIaXNfdRnsu/LWNDzk73C6b4gukYt5f3WPCBc162Pj2a1o6aSedtcjG054LH0ryyKY2Tr63rgzLwJxFyAce6RIREXnCTpeIiMiTg3Z4Odcqd3KHhpfeUNfGy/o9Epvdij6Tt87Hq5w0PkE3tJx8mwAABs9JREFUuQ3HScmZAKy4t4ONa/xQNs4I/0Ut9w5a9x/fxMa1ObzsHZ+fWzqqTw23t19MdR9ccAhWxGbPyr6139q4xchvnbTvRoZx11tusHHsGdB3vRI+KOVPVw9x0qpMn5eDWpaMR7pERESesNMlIiLyhJ0uERGRJwftnO5rq7o4y8MahqeHdy3YbuMeC3elVN4JNV93lk+vEb6uKDZzxM0Lzrdxs3VLUnovMvbXTPzEkaiycmevqlLZxtEnHxGRP0f89WMbL3ihuZN2+NubbTzqqSedtN//5Tobl+ZTi3ikS0RE5Ak7XSIiIk8O2uHlJpe6p5T3f2OAjd9sH945JTrsnI4ekdPSiy5yLw/5oMuLNm70ZM2Myifg2GNX27go6SB+2bBXw4vAykN9iQ520cuMAGDibX1t/P1I9zKysbc/ZOMrmv/exi1Gfoxc4pEuERGRJ+x0iYiIPGGnS0RE5MlBO6dbtHWr+48+4XLvAb+z8fpuiX931F8aXvdR9wV3/H/Dc7ttvKzLy07auM2FNq655Hsb5+uJN+Tfmn17nOUaG/YkyElEvtSYHF5euGBe4suJPr/2QRv3H9k9p3XgkS4REZEn7HSJiIg8OWiHl5OpOSl8eHzhpMzKWNb7KRvHXh7yyPLTbNz0m9Qe2EzlzzXnvpMw7ZdPD3OWW8zI7WUHZFy+pqeNJ7R8P2G+rx44yVnmU4co9nKihxacbuMhp60stfflkS4REZEn7HSJiIg8qZDDy5mIfYg9ED7wOPZM1cYPVfdQo4Pf9jua2nju05WdtOMLwrs/fT3xGBu3GJjZHcYy0b3GKmd5zm6xceF9C5w03p+KqIw54Rhn8bmTxtn4kU1tSu1teaRLRETkCTtdIiIiT9jpEhERecI53RStHFEtYdrA+dc4y01mfFba1akQKv1rvo2vG329k/bpLWNs/O6Jj9p40OlDnXyVc9wWq14+1sanVp/npJ0y/yIbN9j+ZU7fl0I7Bpxo4wktH89jTShqzZ2nOMvV/xPGjceUjUvmKndsZ+Mto7Y7ac2q7LTxtEE9Iim5PU+ER7pERESesNMlIiLyhMPLSejJnW085cSxManhZUHyXn1PNaq4Dp/5o7N8fO9LbTy3+/M2XtvLvVyr5Yzs33v7eeFw5qsnhg+6nrW7wMnX4G5eKuZDqz8uzXcVKLDx6pNtvOiaMU5ah5nhtFtjNylrVZo3c5bXXNwibr7W57h3lrqt+Us2nr3TvSxowMjwLnINPp2VbRUT4pEuERGRJ+x0iYiIPGGnS0RE5AnndJNY372WjVtVcefrok8WqrJLQaWraOEyZ/mI4eFtOSdNamDjKYPuc/KddehNNj7yuk+QiHTrZON1J9d10h6/OXygdYdq4e/U9lMHO/nazZ4Dyr3oJUJA6pcJ9bjuNzZuO4lPFSptVcW9VevSXuGT2OavCveXF8+61sknkbhn669svHxTIyffjGMm2rgS3EsBi6CRtLDEsZtaOfkumh5+JzqO/N5Ja7C29OZxo3ikS0RE5Ak7XSIiIk84vJzErkPDIYvYB9WP/rGjjRs+6WdYgkL7lyy38bNnhQ+ffvwJt52m/eJ+G7/ao5uNX36xt5PvqcHhNQ1dCxI/E+isL863cftHtzppfJKQf21eGWLj2AfT10Ti6QTKjYbjwn3fKduHOGnr++2O+5pnTx7nLJ9QEO5no0/3KXIGnt1LkIo2uncIbD1pb9z3qjbvK2e53Za5Nt4X9xWlj0e6REREnrDTJSIi8oTDy0lcem7i2xmNn3yGjQvB4eV82rdytY0LLjrMSRvS9fc2rnrLDzaed8ODTr72U69LWH6r18OB44IZC21ctHdP2nWl9NWc5A4T953UxcZtwbOSy4o6L8+OWY6fbxSOS7FEd/qmDeYnyJfY/rRfUfp4pEtEROQJO10iIiJP2OkSERF5wjndJF5bFc4dDWuY2wcZU+nYv2GDs1z1ncjyO2HYH92dfO2Q2t2keO8xIsoGj3SJiIg8YadLRETkCYeXk9D3whvp39bMvel647ll8WR0IiIqy3ikS0RE5Ak7XSIiIk/Y6RIREXnCOd0kGj/0sY0XP+Sm1UjxEhMiIqJiPNIlIiLyhJ0uERGRJ6LKe+wQERH5wCNdIiIiT9jpEhERecJOl4iIyBN2ukRERJ6w0yUiIvKEnS4REZEn7HSJiIg8YadLRETkCTtdIiIiT9jpEhERecJOl4iIyBN2ukRERJ6w0yUiIvKEnS4REZEn7HSJiIg8YadLRETkCTtdIiIiT9jpEhERecJOl4iIyBN2ukRERJ6w0yUiIvKEnS4REZEn7HSJiIg8+X+rlKmHCp355gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    ckpt = tf.train.get_checkpoint_state('./')\n",
    "    if ckpt and ckpt.model_checkpoint_path:\n",
    "        saver.restore(sess, ckpt.model_checkpoint_path)\n",
    "        final_pred, acc = sess.run(\n",
    "            [pred, accuracy],\n",
    "            feed_dict={\n",
    "                x: mnist.test.images[:16],\n",
    "                y: mnist.test.labels[:16]\n",
    "            })\n",
    "        orders = np.argsort(final_pred)\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        print(acc)\n",
    "        for idx in range(16):\n",
    "            order = orders[idx, :][-1]\n",
    "            prob = final_pred[idx, :][order]\n",
    "            plt.subplot(4, 4, idx + 1)\n",
    "            plt.axis('off')\n",
    "            plt.title('{}: [{}]-[{:.1f}%]'.format(mnist.test.labels[idx],\n",
    "                                                  order, prob * 100))\n",
    "            plt.imshow(mnist.test.images[idx].reshape((28, 28)))\n",
    "\n",
    "    else:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 结果分析\n",
    "\n",
    "- 对模型结构的理解\n",
    "上面模型使用的是双层神经网络结构，第一层的神经元有100个，第二层的神经元有10个，并且对每一层神经元的weight和bias使用的是截断正态分布函数进行初始化。第一层神经元输出采用的激活函数是relu，第二层神经元输出采用的激活函数是softmax。采用sgd优化器对交叉商损失进行优化。\n",
    "- 对模型训练过程（梯度如何计算，参数如何更新）的理解\n",
    "（1）此模型的梯度下降是通过sgd优化算法来计算的，使用batch_size = 32条数据进行批次计算。\n",
    "（2）参数更新是通过feed_dict函数对之前定义的placeholder图变量进行赋值\n",
    "- 对计算图的理解\n",
    "（1）计算图是指所有的计算操作都被声明为图(graph)中的一个节点(Node)。即使只是声明了一个常量或变量，也并不执行实际的操作，而是向图中增加节点。\n",
    "（2）计算图中的所有数据均以Tensor来存储和表达。Tensor是一个高阶张量，二阶张量为矩阵，一阶张量为向量，0阶张量为标量（即为一个数）。\n",
    "（3）计算图中重复定义同名的变量，并不会覆盖之前定义的变量，每次定义的变量都是向图中追加一个节点。\n",
    "- 解释这⾥的模型为什么效果⽐较差\n",
    "（1）学习率使用的是固定值，没有动态更新\n",
    "（2）隐层神经元个数太少，一般隐层个数=np.sqrt(输入个数 * 输出个数)\n",
    "（3）训练次数太少\n",
    "（4）缺少正则项"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3.0
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}